-- Logs begin at Tue 2022-10-04 09:41:12 UTC, end at Tue 2022-10-04 10:59:14 UTC. -- Oct 04 10:13:32.223960 dv-cinder-859193-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 10:13:32.231262 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 10:13:32 2022] *** Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: nodename: dv-cinder-859193-3-iscsi Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: machine: x86_64 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: clock source: unix Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: pcre jit disabled Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: detected number of CPU cores: 4 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: current working directory: / Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: detected binary path: /usr/bin/uwsgi-core Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: your processes number limit is 47680 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: your memory page size is 4096 bytes Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: detected max file descriptor number: 2048 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: lock engine: pthread robust mutexes Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: thunder lock: enabled Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 10:13:32.238483 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 10:13:32.274714 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: Python main interpreter initialized at 0x5619926ef9c0 Oct 04 10:13:32.274714 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: python threads support enabled Oct 04 10:13:32.274714 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: your server socket listen backlog is limited to 100 connections Oct 04 10:13:32.274714 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: your mercy for graceful operations on workers is 90 seconds Oct 04 10:13:32.275078 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 10:13:32.275148 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: *** Operational MODE: preforking *** Oct 04 10:13:32.275404 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: *** uWSGI is running in multiple interpreter mode *** Oct 04 10:13:32.275404 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: spawned uWSGI master process (pid: 81546) Oct 04 10:13:32.275611 dv-cinder-859193-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 10:13:32.277214 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: spawned uWSGI worker 1 (pid: 81560, cores: 1) Oct 04 10:13:32.277214 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: spawned uWSGI worker 2 (pid: 81561, cores: 1) Oct 04 10:13:32.277214 dv-cinder-859193-3-iscsi devstack@keystone.service[81546]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 10:13:36.666245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:13:36.666245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: from cryptography.utils import int_from_bytes Oct 04 10:13:36.666971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:13:36.666971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: from cryptography.utils import int_from_bytes Oct 04 10:13:36.671637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:13:36.671637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: from cryptography.utils import int_from_bytes Oct 04 10:13:36.676706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:13:36.676706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: from cryptography.utils import int_from_bytes Oct 04 10:13:36.718292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 10:13:36.718481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 10:13:36.718682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] command line args: [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 10:13:36.718960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 10:13:36.719146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ================================================================================ {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 10:13:36.719458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] admin_token = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.719699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] config_dir = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.719894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.720082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] config_source = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.720266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] debug = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.720468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.721753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.721753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.721753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.721753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.721753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.722060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_config_append = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.722120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 10:13:36.722185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.722243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 10:13:36.722300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_dir = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.722360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] command line args: [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 10:13:36.722425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_file = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.722618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 10:13:36.722731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.722875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ================================================================================ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 10:13:36.722963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] admin_token = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config_dir = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] config_source = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] debug = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.723921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724752 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.724974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_config_append = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_dir = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.725959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] publish_errors = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.726939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.727009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.727108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.727197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.727599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.727840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.727965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] standard_threads = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.728932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] use_journal = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] use_json = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] use_stderr = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] use_syslog = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.734995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.734995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.735179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.735179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] publish_errors = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.735323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.735400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.735478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.735550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.735626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.735701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.735780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.735850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.735936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.736053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.736131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.736247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] standard_threads = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.736327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.736398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.736547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.736628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.736628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.736759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.736835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_journal = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.736938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_json = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.737019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_stderr = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.737154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] use_syslog = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.737292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:13:36.737438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.737947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.738969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.739905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.740991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.741996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.742557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.743320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.743479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.743593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.743740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.743866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.743967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.744794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.745956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.746998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.747997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.748988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.749899 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.750925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.751875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.752839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.753973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.754942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.755971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.external = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.756952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.password = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] auth.token = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.757978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.758056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.758129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.758236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.758389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.759878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.external = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.760898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.password = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] auth.token = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.761863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.762950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.763973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.764849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.765936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.766835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.767885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.768833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.769994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.770966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.771927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.772997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.773865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.774916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.775869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.776939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.777940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.778966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.779895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.780900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.781845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.782830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.783751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.784909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.785922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.786991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.787941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.788927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.789878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.790897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] role.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] role.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.791857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.792844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.793853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.794991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.795909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.796946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.797032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.797193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.797273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.803956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.804996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.805989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.806925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.807920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.connection = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.808999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.809948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81560) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.810988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.811513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.811513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.811654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.811786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.812027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:13:36.812266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81561) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 10:13:36.840323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.840562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.840787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.850690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.850937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.851160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.851679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.862226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.890731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.890899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.892669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.892927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.945222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:36.946902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.008810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.009170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.009994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.010237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.167764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.167934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.168125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.168347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.168567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.168786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.168895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.169063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.169293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.169518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.169763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.169961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.194845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.194988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.195781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.196187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.196318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.196440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.196662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.196990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.207519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.208228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.220348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.220582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.220695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.220804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.220906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81561) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.221923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81560) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:13:37.239313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.239517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:13:37.239858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.240056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:13:37.240332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.240596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.240894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.241135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.245081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.245408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:13:37.245780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.245982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:13:37.246259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.246525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.246789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.247051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.252639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.252947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.253224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.253515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.260794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.261760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.262047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.262324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.262609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.263689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.263972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.264255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.264542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.264830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.265119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.265392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.265669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.265943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.266222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.266492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.269550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.272415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.272703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.273022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.273312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.273716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.273864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.274587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.274879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.275162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.275451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.275727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.289182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.291035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.291315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.295812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.296096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.296379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.299091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.300552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.300839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.302763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.303050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.305978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.306271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.306578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.307059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.307352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.307637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.307922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.308221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.310256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.310550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.310840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.312840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.313152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.318830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.319133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.319430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.319724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.320006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.320289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.320565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.321149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.326342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.326620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.330395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.333444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.333742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.337551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.337927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.338838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.341804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.341942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.344920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.345060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.345221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.345353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.345507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.345793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.346071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.346455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.349492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.353184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.357303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.357303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.357303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.357303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.357618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.357881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.362132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.362419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.362686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.362949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.370215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.371867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.373411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.373689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.374499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.374790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.375059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.375324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.376307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.376593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.376903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.377189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.377471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.382128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.384167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.385731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.386012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.388645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.388733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.388957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.389045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.389245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.389331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.389529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.389614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.389830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.390174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.390270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.390386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.401913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.402200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.402484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.402771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.403154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.403450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.403735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.406186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.409210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.412025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.413572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.415281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.415576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.415869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.423023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.423213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.423293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.423737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.425351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.426617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.426789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.426908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.427193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.427467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.428455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.428738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.429173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.436035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.436322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.436458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.436601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.437947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.440835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.441264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.441565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.441838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.443407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81561) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.443691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.443978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.444254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.444525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.444798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.445092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.445380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.446171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.446308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.446572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.446848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.447135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.447379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81561) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.450392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.450692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.450990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.457822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81560) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:13:37.458108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.458395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.458679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.458953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.459228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.459514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.459795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.460068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.460349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.460633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.460938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.461221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.462137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81560) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:13:37.481460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5619926ef9c0 pid: 81561 (default app) Oct 04 10:13:37.484849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e8855-02bc-4092-bdcb-5b8c78c78dde None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:37.485152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e8855-02bc-4092-bdcb-5b8c78c78dde None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:37.485427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e8855-02bc-4092-bdcb-5b8c78c78dde None None] PATH_INFO: `/v3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:37.489589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Tue Oct 4 10:13:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:37.497663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5619926ef9c0 pid: 81560 (default app) Oct 04 10:13:45.364453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:45.364981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:45.365092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:45.366352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 2/2] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:45.374627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:45.375067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:45.375187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:45.445416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: INFO dbcounter [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] Registered counter for database keystone Oct 04 10:13:45.466540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG oslo_db.sqlalchemy.engines [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81560) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 10:13:45.472711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writer thread running {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 10:13:45.653634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG passlib.handlers.bcrypt [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81560) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 10:13:45.653946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG passlib.handlers.bcrypt [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81560) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 10:13:45.711674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:45.713301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:45.841712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1/3] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 04 10:13:46.222846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:46.295676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: INFO dbcounter [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] Registered counter for database keystone Oct 04 10:13:46.315450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG oslo_db.sqlalchemy.engines [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81561) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 10:13:46.322378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writer thread running {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 10:13:46.445127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cc830e-4b0d-4925-a171-fc36e254aa2b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:46.450190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:46.450813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:46.451095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:46.451369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:46.459467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:46.512395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING keystone.common.rbac_enforcer.enforcer [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 10:13:46.512736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d5f1a-2b61-4651-ac55-6c7a6a3370ed admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:46.522034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 3/4] 10.222.0.42 () {58 vars in 1324 bytes} [Tue Oct 4 10:13:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:13:46.528643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:46.545622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4ce84-39ad-48ff-9aa4-c205eb7baa40 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:46.549965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:46.551198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:46.551477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:46.551748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] PATH_INFO: `/v3/projects/76254fdd7dd646c69803abc10a472bc5` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:46.553273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] RBAC: Authorizing `identity:get_project(project_id=76254fdd7dd646c69803abc10a472bc5)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:46.606774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 10:13:46.607121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb27499-d0d9-4b18-a8db-dfca8d655f0a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:46.611908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 2/5] 10.222.0.42 () {56 vars in 1207 bytes} [Tue Oct 4 10:13:46 2022] GET /identity/v3/projects/76254fdd7dd646c69803abc10a472bc5 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:48.751168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3565a-04ec-45e4-877c-44e00ad971ca None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.751719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3565a-04ec-45e4-877c-44e00ad971ca None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.751719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3565a-04ec-45e4-877c-44e00ad971ca None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.755871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 4/6] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:48.761497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.761869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.762058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.782712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:48.784236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:48.793278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 3/7] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 10:13:49.066710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3565a-04ec-45e4-877c-44e00ad971ca None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:49.079594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3565a-04ec-45e4-877c-44e00ad971ca None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:49.083826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ead676-141d-4f61-b304-2631ae9bc42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:49.084433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ead676-141d-4f61-b304-2631ae9bc42c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:49.084718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ead676-141d-4f61-b304-2631ae9bc42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:49.085037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ead676-141d-4f61-b304-2631ae9bc42c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:49.092653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ead676-141d-4f61-b304-2631ae9bc42c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:49.094478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ead676-141d-4f61-b304-2631ae9bc42c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:49.103677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 5/8] 10.222.0.42 () {58 vars in 1324 bytes} [Tue Oct 4 10:13:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:13:49.109038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:49.118556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e38a4-e24e-4d50-89a3-184327524b7d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:49.122817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05470d-e874-4940-9766-49e533c59616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:49.123464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05470d-e874-4940-9766-49e533c59616 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:49.123741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05470d-e874-4940-9766-49e533c59616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:49.124013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05470d-e874-4940-9766-49e533c59616 admin admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:49.125349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05470d-e874-4940-9766-49e533c59616 admin admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:49.127164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05470d-e874-4940-9766-49e533c59616 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:49.128663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 4/9] 10.222.0.42 () {56 vars in 1198 bytes} [Tue Oct 4 10:13:49 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:51.393696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68bc5-42dd-4f36-bba3-5e298b787346 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:51.394223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68bc5-42dd-4f36-bba3-5e298b787346 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:51.394329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68bc5-42dd-4f36-bba3-5e298b787346 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:51.394505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eb30d-349d-4980-8c18-aa13b5c04e6a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:51.394883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eb30d-349d-4980-8c18-aa13b5c04e6a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:51.395065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eb30d-349d-4980-8c18-aa13b5c04e6a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:51.395569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 6/10] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:51.396392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 5/11] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:13:51.401298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:51.401580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:51.401858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:51.402126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:51.402413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:51.402690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:51.420351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:51.421591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:51.443207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG passlib.handlers.bcrypt [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81561) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 10:13:51.443619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG passlib.handlers.bcrypt [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81561) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 10:13:51.493499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:51.495079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:51.502432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:13:51.593334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:13:51.775589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:51.800688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:51.864116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:51.882797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72fc2a4-ed26-47cf-bf51-529c2ee4732b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:51.921371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:51.922106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:51.922449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:51.922789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:51.973519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add31f4-50f4-4273-8c38-278b93d07b80 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:51.975730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:51.990247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:51.991737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.011140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.012237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.012516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.012795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.015379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:52.024899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:13:52.034295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:13:52.035318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:13:52.036704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 8/14] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:13:52.045553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:52.092847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 7/15] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:13:52.098384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:52.140443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18511a-0bcb-48a3-ab69-66f412366217 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:52.173228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.173828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.174104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.174375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.175035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:13:52.175332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:52.176774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.189122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 9/16] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:13:52.195486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:52.197784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fe568-5b51-44aa-a040-5a5ad534375a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:52.227394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:52.229454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:52.245443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.246358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.246778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.247099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.259538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:52.261045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:13:52.278908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:13:52.279816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:13:52.280610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:13:52.280610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:13:52.280610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:13:52.280610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 8/17] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:13:52.285948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:52.300235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a13c6-7bc3-4885-990b-39f83895f483 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:52.333539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.334168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.334443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.334724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.337073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:52.338531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.340203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:13:52 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:13:52.345599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:52.386286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112cd6f-1668-4e73-86f9-7b089be0046f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:52.419325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.419950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.420208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.420485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.421158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:13:52.421463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:52.422858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.439553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:52.440426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:13:52.446822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362703f-d3c4-4683-9098-3980fdd9d3cb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:52.487516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec4983-7031-4bb4-8587-6fca1b97c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.488161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec4983-7031-4bb4-8587-6fca1b97c2be None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.488444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec4983-7031-4bb4-8587-6fca1b97c2be None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.488718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec4983-7031-4bb4-8587-6fca1b97c2be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.489939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec4983-7031-4bb4-8587-6fca1b97c2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:52.491447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec4983-7031-4bb4-8587-6fca1b97c2be None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.493276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:13:52.499486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:52.537742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:52.609219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06642a2a-e2ff-4a30-949e-a81cda0d6a61 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:52.644779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:52.645434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:52.645716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:52.645991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:52.646718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.domain.id=default, user.id=105ad03a7a5c40c78500f0ee01583fae) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:13:52.647029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:52.649037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3d40-73c8-424e-8477-07caaa756fe5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:52.656096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1322 bytes} [Tue Oct 4 10:13:52 2022] GET /identity/v3/role_assignments?user.id=105ad03a7a5c40c78500f0ee01583fae&scope.domain.id=default&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:55.060174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabb6b8-83d1-4307-bc6e-6861991f57dd None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.060720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabb6b8-83d1-4307-bc6e-6861991f57dd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.060720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabb6b8-83d1-4307-bc6e-6861991f57dd None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.062336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:55.067713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38f626-013a-4661-a321-0aa7d095c6d7 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.068113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38f626-013a-4661-a321-0aa7d095c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.068303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38f626-013a-4661-a321-0aa7d095c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.092716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5d38f626-013a-4661-a321-0aa7d095c6d7 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:55.093971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5d38f626-013a-4661-a321-0aa7d095c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:55.193056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:13:55.237279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10455eaa-e644-43e6-a2fa-99a4a27197e7 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.237614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10455eaa-e644-43e6-a2fa-99a4a27197e7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.237833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10455eaa-e644-43e6-a2fa-99a4a27197e7 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.239159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:55.244706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305e14-f56b-4621-8ef8-a500c723be13 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.245030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305e14-f56b-4621-8ef8-a500c723be13 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.245311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3305e14-f56b-4621-8ef8-a500c723be13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.271908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d3305e14-f56b-4621-8ef8-a500c723be13 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:55.273141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d3305e14-f56b-4621-8ef8-a500c723be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:55.307882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:55.357165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 12/25] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:13:55.510373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffdf6c-fc7f-444c-8bb8-67c8f9c89a0f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.511032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffdf6c-fc7f-444c-8bb8-67c8f9c89a0f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.511513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffdf6c-fc7f-444c-8bb8-67c8f9c89a0f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.513105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:55.518896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375f21-d001-4698-b059-b78bd5930847 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.519198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375f21-d001-4698-b059-b78bd5930847 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.519481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0375f21-d001-4698-b059-b78bd5930847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.545858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b0375f21-d001-4698-b059-b78bd5930847 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:55.547061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0375f21-d001-4698-b059-b78bd5930847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:55.626862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:13:55.795613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffdf6c-fc7f-444c-8bb8-67c8f9c89a0f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:55.808163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5d18a-d973-4590-afd4-9a53c41fd445 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:55.808443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5d18a-d973-4590-afd4-9a53c41fd445 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:55.808726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5d18a-d973-4590-afd4-9a53c41fd445 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:55.810325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 14/28] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:55.810824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5d18a-d973-4590-afd4-9a53c41fd445 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:55.965448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:55.969034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffdf6c-fc7f-444c-8bb8-67c8f9c89a0f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:55.980158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5d18a-d973-4590-afd4-9a53c41fd445 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.009673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.010425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.010803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.011084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.019248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab4312-791f-4cf8-aec1-b865ca727250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.019891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab4312-791f-4cf8-aec1-b865ca727250 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.020172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab4312-791f-4cf8-aec1-b865ca727250 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.020447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab4312-791f-4cf8-aec1-b865ca727250 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.032964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab4312-791f-4cf8-aec1-b865ca727250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:13:56.033145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.034449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-c6ab4312-791f-4cf8-aec1-b865ca727250 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:13:56.040482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:13:56.041421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:13:56.042795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 15/29] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:13:56.043414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.043414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.043840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:13:56.051654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:13:56.053750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:13:56.055049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:13:56.055049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:13:56.055049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:13:56.055049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 15/30] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:13:56.055049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.064343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:56.065655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:56.076676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:56.153674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 16/31] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:13:56.154675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.180587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71cc84-c7c3-4b5d-94d3-8627d8cd8726 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.252973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45adba-6bbc-4cb0-b573-0465debb034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.253714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45adba-6bbc-4cb0-b573-0465debb034b None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.254605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45adba-6bbc-4cb0-b573-0465debb034b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.254849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45adba-6bbc-4cb0-b573-0465debb034b None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.255530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45adba-6bbc-4cb0-b573-0465debb034b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.256928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45adba-6bbc-4cb0-b573-0465debb034b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.280096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:56.302559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cd76f-ac6a-41b6-b1b3-6070e8af7088 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.337948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.338561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.338834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.339116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.339787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:13:56.340112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.341539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.354051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 17/32] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:56.354613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.463632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411cbc-075c-44d1-80ac-b8f3c5309f91 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.485020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 16/33] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Oct 4 10:13:56 2022] POST /identity/v3/roles => generated 225 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:13:56.486282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651caa1-dfd8-4910-913a-feea086274e8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.486574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651caa1-dfd8-4910-913a-feea086274e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.486895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651caa1-dfd8-4910-913a-feea086274e8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.488255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 17/34] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:56.488810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651caa1-dfd8-4910-913a-feea086274e8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.507007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.508458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.508888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.509274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.510161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:13:56.510692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.512305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d447b0-8d9e-437a-933a-c46229233f73 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.522415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:56.528840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 18/35] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:56.530401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.530800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.531294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.561274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:56.562684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:56.595099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:56.626002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651caa1-dfd8-4910-913a-feea086274e8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.673159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.673984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.674360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.674731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.676206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 19/36] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:13:56.677324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.681155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.682937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.689430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 18/37] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:13:56 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:56.689430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.828974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4756284-2bd3-4b5c-8aad-43bd61346ff1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.833283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0e92f-6b6e-4e38-92c2-6aa149f296a3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:56.864171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:56.871595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7308f-64a0-4d29-ad03-eaccfa41528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.872213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7308f-64a0-4d29-ad03-eaccfa41528c None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.872499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7308f-64a0-4d29-ad03-eaccfa41528c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.872777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7308f-64a0-4d29-ad03-eaccfa41528c None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.873602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7308f-64a0-4d29-ad03-eaccfa41528c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.875338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7308f-64a0-4d29-ad03-eaccfa41528c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.879593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:56.880458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:56.881149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:56.881989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:56.883220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:56.884930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:56.973070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 19/38] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Oct 4 10:13:56 2022] POST /identity/v3/roles => generated 219 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:13:56.974589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:56.977744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:57.112813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d934917-98ee-46d6-b077-8edadd2d7b69 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:57.152941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:57.153609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.153899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.154183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.155504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:57.157022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:57.158675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 20/39] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:57.159450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:57.208545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 20/40] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:13:56 2022] POST /identity/v3/projects => generated 301 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:13:57.210620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a5633-2982-4d5c-9689-92e9b0ce1c2c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.210963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a5633-2982-4d5c-9689-92e9b0ce1c2c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.211247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a5633-2982-4d5c-9689-92e9b0ce1c2c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.212730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 21/41] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:57.213459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a5633-2982-4d5c-9689-92e9b0ce1c2c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:57.247598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:57.297217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf89b8c-75ef-45f7-a361-6396242901e4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:57.337374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb355498-c0ca-4a87-8b9d-14a1b33f461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:57.338003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355498-c0ca-4a87-8b9d-14a1b33f461e None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.338295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355498-c0ca-4a87-8b9d-14a1b33f461e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.338568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355498-c0ca-4a87-8b9d-14a1b33f461e None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.339343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb355498-c0ca-4a87-8b9d-14a1b33f461e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:57.340100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a5633-2982-4d5c-9689-92e9b0ce1c2c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:57.340758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb355498-c0ca-4a87-8b9d-14a1b33f461e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:57.384290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:57.385037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.385323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.385607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] PATH_INFO: `/v3/domains/default/users/105ad03a7a5c40c78500f0ee01583fae/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.393744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=105ad03a7a5c40c78500f0ee01583fae, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:57.395333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:57.415062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 21/42] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Oct 4 10:13:57 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:13:57.416337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.416675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.416982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.437820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:57.447261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:57.518362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:13:57.533643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 22/44] 10.222.0.42 () {58 vars in 1361 bytes} [Tue Oct 4 10:13:57 2022] PUT /identity/v3/domains/default/users/105ad03a7a5c40c78500f0ee01583fae/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:13:57.788541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:57.855237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:57.895557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3185a35-e541-40e4-996d-337b5febc7b6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:57.930036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c575d-02e2-4544-acb1-2d311369877e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:57.930676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c575d-02e2-4544-acb1-2d311369877e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.930960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c575d-02e2-4544-acb1-2d311369877e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.931237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c575d-02e2-4544-acb1-2d311369877e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.932343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c575d-02e2-4544-acb1-2d311369877e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:57.933838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c575d-02e2-4544-acb1-2d311369877e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:57.935392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 23/45] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:13:57.941090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:57.985957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:58.042572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b0865-3c28-4128-92a5-91082e1957d3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:58.076316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a7e69-8d89-4ef1-8543-c25903306877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:58.076974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a7e69-8d89-4ef1-8543-c25903306877 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:58.077265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a7e69-8d89-4ef1-8543-c25903306877 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:58.077546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a7e69-8d89-4ef1-8543-c25903306877 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:58.078220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a7e69-8d89-4ef1-8543-c25903306877 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:58.079680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a7e69-8d89-4ef1-8543-c25903306877 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:58.156108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:13:57 2022] POST /identity/v3/projects => generated 312 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:13:59.700637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271ecda-667f-4ffa-a64a-9db19f59b016 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:59.701205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271ecda-667f-4ffa-a64a-9db19f59b016 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:59.701205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271ecda-667f-4ffa-a64a-9db19f59b016 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:59.702560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:59.707947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:59.708359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:59.708522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:59.727347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:59.728880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:59.795071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:00.065347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271ecda-667f-4ffa-a64a-9db19f59b016 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.166376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271ecda-667f-4ffa-a64a-9db19f59b016 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.196959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:00.205239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc20d9-e4d0-4b60-b539-2c5247a809e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.206055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc20d9-e4d0-4b60-b539-2c5247a809e0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.206340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc20d9-e4d0-4b60-b539-2c5247a809e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.206613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc20d9-e4d0-4b60-b539-2c5247a809e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.217327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc20d9-e4d0-4b60-b539-2c5247a809e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a8fc20d9-e4d0-4b60-b539-2c5247a809e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:00.222176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:00.222985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:00.223769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:00.229762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.254220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:00.291898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314ef9e-0c67-4668-81f3-6a74aed20fc3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.292176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314ef9e-0c67-4668-81f3-6a74aed20fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.292456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314ef9e-0c67-4668-81f3-6a74aed20fc3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.293970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 26/50] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:00.298687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.299176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.299548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.318692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:00.319805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:00.330334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82937373-e3c3-4e7f-8344-0cbc8deea4ad None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.363637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.364245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.364526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.364805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.365503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:00.365819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:00.368468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:00.376487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 25/51] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:00.382439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.385642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 27/52] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:00.467887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:00.482630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e96d5d-5152-456b-b2f2-ce50b0b75527 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.516280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.516936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.517228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.517505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.518644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:00.520108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:00.521716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:00 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:00.527409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.631877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef01aec-4624-40a0-b536-98d88a260b0f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.649984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:00.652906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.667090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.667726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.668006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.668285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.669424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:00.670875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:00.672592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 28/54] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:00.678151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.752772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa7896-f275-4465-ae2a-749fb09147e6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.780327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d01b8-8f09-425a-b29f-273b4adda262 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.786748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.787380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.787660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.787940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.789836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:00.791237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:00.792784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:00.798226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:00.814938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.815541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.815819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.816317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.816806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.domain.id=default, user.id=105ad03a7a5c40c78500f0ee01583fae) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:00.817140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:00.818572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a56873-576c-486a-9333-52a56c44b9b1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:00.821971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:00.827519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 29/56] 10.222.0.42 () {56 vars in 1322 bytes} [Tue Oct 4 10:14:00 2022] GET /identity/v3/role_assignments?user.id=105ad03a7a5c40c78500f0ee01583fae&scope.domain.id=default&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:00.903928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad270e48-3f0a-4304-98a5-c45e41fcd14b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:00.939602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ceb5b1-ff75-4c7d-b5d3-0c511137a315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:00.940227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceb5b1-ff75-4c7d-b5d3-0c511137a315 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:00.940509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceb5b1-ff75-4c7d-b5d3-0c511137a315 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:00.940786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceb5b1-ff75-4c7d-b5d3-0c511137a315 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:00.941477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ceb5b1-ff75-4c7d-b5d3-0c511137a315 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:00.942938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ceb5b1-ff75-4c7d-b5d3-0c511137a315 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:01.070872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 28/57] 10.222.0.42 () {60 vars in 1140 bytes} [Tue Oct 4 10:14:00 2022] POST /identity/v3/projects => generated 298 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:14:03.227168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d826563-909f-42b9-b984-ecca9a3e2db6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:03.227689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d826563-909f-42b9-b984-ecca9a3e2db6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:03.227689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d826563-909f-42b9-b984-ecca9a3e2db6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:03.229101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 30/58] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:03.234387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:03.234815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:03.234952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:03.253321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:03.254530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:03.319195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 29/59] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:03.599265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d826563-909f-42b9-b984-ecca9a3e2db6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:03.650866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:03.699518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d826563-909f-42b9-b984-ecca9a3e2db6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:03.731744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706f0c9-eae2-40df-9f6f-852d3863a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:03.732365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f0c9-eae2-40df-9f6f-852d3863a795 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:03.732647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f0c9-eae2-40df-9f6f-852d3863a795 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:03.732944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f0c9-eae2-40df-9f6f-852d3863a795 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:03.734045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706f0c9-eae2-40df-9f6f-852d3863a795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:03.735498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706f0c9-eae2-40df-9f6f-852d3863a795 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:03.737054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 31/60] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:03.742799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:03.745825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:03.839962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8323518-5dcb-4f8a-96d5-43db19ba2191 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:03.872646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f654f05-9851-42f4-9a70-f65df8d409ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:03.873222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f654f05-9851-42f4-9a70-f65df8d409ed None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:03.873504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f654f05-9851-42f4-9a70-f65df8d409ed None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:03.873777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f654f05-9851-42f4-9a70-f65df8d409ed None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:03.874414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f654f05-9851-42f4-9a70-f65df8d409ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:03.875853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f654f05-9851-42f4-9a70-f65df8d409ed None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:03.962116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 30/61] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:03 2022] POST /identity/v3/users => generated 275 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:07.219829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d703-8113-4e32-953c-32c86920e92a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.220373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d703-8113-4e32-953c-32c86920e92a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.220373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d703-8113-4e32-953c-32c86920e92a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.225495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 32/62] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:07.225624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf20961-fb7b-462b-869a-93dac35083ba None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.225624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf20961-fb7b-462b-869a-93dac35083ba None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.225624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf20961-fb7b-462b-869a-93dac35083ba None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.227050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 31/63] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:07.233175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d305c-2599-421c-80b5-cce0cf4fc08d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.233951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d305c-2599-421c-80b5-cce0cf4fc08d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.234575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d305c-2599-421c-80b5-cce0cf4fc08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.244124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82fecd-34cc-4ca7-b76f-833e7d090f6d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.245495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82fecd-34cc-4ca7-b76f-833e7d090f6d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.246002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82fecd-34cc-4ca7-b76f-833e7d090f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.263626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-640d305c-2599-421c-80b5-cce0cf4fc08d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:07.265983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-640d305c-2599-421c-80b5-cce0cf4fc08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:07.283169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7c82fecd-34cc-4ca7-b76f-833e7d090f6d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:07.284658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7c82fecd-34cc-4ca7-b76f-833e7d090f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:07.421408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:07.431060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 32/64] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:07.536521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 33/65] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:07.708584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd5b98-6fef-4f33-9362-06106cb6c564 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.708918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd5b98-6fef-4f33-9362-06106cb6c564 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.709160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd5b98-6fef-4f33-9362-06106cb6c564 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.710876 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 33/66] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:07.716168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0649b2-3c93-435c-be97-63d90a3ac2b8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.716456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0649b2-3c93-435c-be97-63d90a3ac2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.716736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0649b2-3c93-435c-be97-63d90a3ac2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.741019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ee0649b2-3c93-435c-be97-63d90a3ac2b8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:07.742332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ee0649b2-3c93-435c-be97-63d90a3ac2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:07.754813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd5b98-6fef-4f33-9362-06106cb6c564 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:07.830288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:07.855319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:07.856785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe8bd-78a5-4d30-9640-b27c3584b57e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.857106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe8bd-78a5-4d30-9640-b27c3584b57e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.857383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbe8bd-78a5-4d30-9640-b27c3584b57e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.858978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 35/68] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:07.864452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.864743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.865063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.895544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:07.896756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:07.898405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd5b98-6fef-4f33-9362-06106cb6c564 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:07.939667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:07.940306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:07.940587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:07.940881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:07.957819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:07.963532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:07.964388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:07.965189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:07.972149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 34/69] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:07.973421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.001062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 36/70] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:08.001716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.049744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.139917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1a7b5-2533-45a2-b024-0970f550ecca None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.150147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.153051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47cb9e-d609-4d9e-b2b9-41260aa51475 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.178410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.179572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.179868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.180125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.180844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:08.181229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:08.182610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:08.196736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 37/71] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:08.201989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.213815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.214755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.215251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.215727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.226711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:08.232281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:08.233149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:08.233929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:08.236181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 35/72] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:08.242512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.316359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d4fa2-9230-4267-ae07-aa4918b87a28 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.347605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc8f2-7cab-4a3b-858d-96dbd64d833e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.353340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.354151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.354581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.355081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.358743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.367231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:08.372476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:08.381645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 38/73] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:08.382985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.383953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c75c8-6613-42d1-bc44-ed2603238312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.384530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c75c8-6613-42d1-bc44-ed2603238312 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.384813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c75c8-6613-42d1-bc44-ed2603238312 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.385115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c75c8-6613-42d1-bc44-ed2603238312 None admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.386253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c75c8-6613-42d1-bc44-ed2603238312 None admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:08.387710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c75c8-6613-42d1-bc44-ed2603238312 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:08.389312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 36/74] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:08.390597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97cc5d-e903-413a-b406-92073f649c93 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.390826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97cc5d-e903-413a-b406-92073f649c93 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.391103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97cc5d-e903-413a-b406-92073f649c93 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.392289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 37/75] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:08.394587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97cc5d-e903-413a-b406-92073f649c93 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.412369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.492642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dd19a-b191-41fa-a0d2-6be7dcb1064b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.498486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97cc5d-e903-413a-b406-92073f649c93 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.527185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.527794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.528074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.528401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.529102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:08.529402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:08.530789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:08.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.534080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.534364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.534640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.539468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 39/76] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:08.540268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.547151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:08.552007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:08.552945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:08.553746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 38/77] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:08.554965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.597431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.636776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.645917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41a0a2-b8ad-4de6-bba7-465863a51342 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.660648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1a722-4929-4d9e-a254-205e156d9fb0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.679820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.680428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.680712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.681026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.681693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:08.682002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:08.683377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d96ce3-953d-4386-9d9a-7375960f6d38 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:08.691508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 40/78] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:08.692619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.692937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.693221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.701635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.702421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.702707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.702984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.711699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:08.712913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:08.770144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:08.771628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:08.778581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 41/79] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:08.779428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.815008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.878361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.879998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 39/80] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:08.879998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.902518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb503906-468b-4292-bb3e-0d8a047a1830 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:08.938315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:08.939050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:08.939239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:08.939521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:08.956333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:08.957827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:08.965022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 42/81] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:08.965022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:08.983966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2081b5e-fb77-4dc6-b2b3-8435233a06cb None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.022330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.023883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.024165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.024699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.025659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:09.025977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.028074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.034831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.045734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 40/82] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.045734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.069237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abf212-92e0-4e63-aa76-9f74d34ef3e5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.105192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.105826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.106109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.106388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.107523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.109006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.110571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 43/83] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:08 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.111338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.124791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.146975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce464cd-572c-40f9-bd55-b66ed2747a02 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.180100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.180496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.180777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.181550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.182688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.184141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.185779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 41/84] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.190015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.212213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a424ca3-0472-4714-ad9c-0fa2cf56d0d7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.246510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.247118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.247405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.247685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.255666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.290862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64275f-ac7b-48ba-b003-98800693a5db None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.316728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.318231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.325454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.326083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.326364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.326641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.327805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.329181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.329285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.330879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 42/85] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.335575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.419814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 44/86] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:09.420677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.436710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06667cb9-df59-4465-afed-17bc8670648c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.468665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.469319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.469596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.469874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.470965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.472417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.474004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 43/87] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.474004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.480711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.520807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de16450-ca14-4e86-b421-68eb6aec96a4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.531765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.556074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.556718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.557169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.557443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.575385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144177d-9ded-4b04-9ce9-7dcbfd6aad0a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.608956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.609569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.610139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.610139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.610888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=105ad03a7a5c40c78500f0ee01583fae) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:09.611204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.612664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.618632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 44/88] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/role_assignments?user.id=105ad03a7a5c40c78500f0ee01583fae&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.619513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.638863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.640395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.694845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.733101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13192d5a-e579-4b52-bf82-97d299c7cc10 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.760520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 45/89] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:09.762827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.772260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.773493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.773778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.774060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.780586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.852461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.854357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.885070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422a09b-2c5e-40fd-862d-5f0b15b3937c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:09.919284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:09.924353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:09.925202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:09.930659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:09.930659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:09.930659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:09.930659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:09.931135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 46/90] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:09.932179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:09.959702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 45/91] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:09.960879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:10.028668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:10.052132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed3072-88c4-4385-b956-c9f993504f8d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:10.058246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedbe78-0e79-45e4-b7f5-fd176f4b57f2 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:10.093531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:10.094255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:10.094603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:10.094887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:10.095167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:10.095356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:10.095458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:10.095851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:10.096544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:10.096800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:10.098026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:10.098561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:10.099640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 46/92] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:09 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:10.100400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:10.139114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:10.201721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b061-f623-44af-b9fb-00d156e2f4de None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:10.219684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 47/93] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:09 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:10.220749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:10.250941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:10.252146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:10.252655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:10.253212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:10.254167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=135d762e2a5b413aa4e27f4cc35ec7e6) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:10.254700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:10.256814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:10.269389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 47/94] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:10 2022] GET /identity/v3/role_assignments?user.id=135d762e2a5b413aa4e27f4cc35ec7e6&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=76c32cb614c145319ad750282d6d3551 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:10.270838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:10.344877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c0c5f-487d-4018-8992-83ce2fb54bf0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:10.411658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:10.412330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:10.412613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:10.412915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] PATH_INFO: `/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:10.421882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] RBAC: Authorizing `identity:get_project(project_id=c0d011eb1d8d4352b06cfda0ed150bb9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:10.424591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:10.427548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 48/95] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:10 2022] GET /identity/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:10.436574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:10.442310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:10.451001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:10.454709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7bf09-710c-4122-9a13-2744db31a0b7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:10.499238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:10.499877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:10.500242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:10.500587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:10.501416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] RBAC: Adding query filter params (role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=135d762e2a5b413aa4e27f4cc35ec7e6) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:10.501729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:10.503395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586d7c-361d-4bf4-974b-c860e5929dbd None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:10.511238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 48/96] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:10 2022] GET /identity/v3/role_assignments?user.id=135d762e2a5b413aa4e27f4cc35ec7e6&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=97b084644adb4949ab768912c0bb75e8 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:10.584574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5749d-bdca-4ff3-a14d-7b4187bbbe84 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:10.630588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:10.631203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:10.631482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:10.631757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:10.632563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=c0d011eb1d8d4352b06cfda0ed150bb9, user.id=135d762e2a5b413aa4e27f4cc35ec7e6) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:10.632895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:10.634533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc19070-38fe-4f88-af89-a38625878669 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:10.640698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 49/97] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:10 2022] GET /identity/v3/role_assignments?user.id=135d762e2a5b413aa4e27f4cc35ec7e6&scope.project.id=c0d011eb1d8d4352b06cfda0ed150bb9&role.id=76c32cb614c145319ad750282d6d3551 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:12.019700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb1b47-fcea-46c2-9bbf-069e4ef90a15 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:12.020302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb1b47-fcea-46c2-9bbf-069e4ef90a15 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:12.020302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb1b47-fcea-46c2-9bbf-069e4ef90a15 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:12.021926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 49/98] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:12.027768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:12.028209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:12.028410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:12.054386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:12.055624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:12.131548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:12.140045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 50/99] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:12.692406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb1b47-fcea-46c2-9bbf-069e4ef90a15 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:12.818533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb1b47-fcea-46c2-9bbf-069e4ef90a15 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:12.862490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:12.863133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:12.863414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:12.863691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:12.875160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:12.879912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:12.880771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:12.881575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 50/100] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:12.887133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:13.031301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06edda-d308-49cf-b9c2-8d46397c182f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:13.081758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:13.082423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.082704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.082977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.083642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:13.083941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:13.085910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:13.094374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 51/101] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:13.099861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:13.136138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:13.249036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316bcbd-4b44-44f1-8e80-363cd258384f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:13.302737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba766d2d-6f90-4834-99be-ff52b291a043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:13.303379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba766d2d-6f90-4834-99be-ff52b291a043 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.303662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba766d2d-6f90-4834-99be-ff52b291a043 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.304455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba766d2d-6f90-4834-99be-ff52b291a043 None admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.309535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba766d2d-6f90-4834-99be-ff52b291a043 None admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:13.311000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba766d2d-6f90-4834-99be-ff52b291a043 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:13.312763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 51/102] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:13 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:13.318161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:13.357182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae033428-fcbd-4f07-b83d-6b5bafec6b74 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.357462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae033428-fcbd-4f07-b83d-6b5bafec6b74 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.357912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae033428-fcbd-4f07-b83d-6b5bafec6b74 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.359570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 52/103] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:13.374397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd444df-a88e-48cc-861b-e248a18bd470 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.374873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd444df-a88e-48cc-861b-e248a18bd470 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.375294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd444df-a88e-48cc-861b-e248a18bd470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.414314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cbd444df-a88e-48cc-861b-e248a18bd470 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:13.416566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cbd444df-a88e-48cc-861b-e248a18bd470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:13.429660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:13.495143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17a857-6adb-43fa-a0ea-e6f23fe56ade None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:13.565714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:13.566391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.566677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.566958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.568237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:13.569863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:13.571915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 52/104] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:13 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:13.578615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:13.602532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 53/105] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:13.700441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e00ba-8c1a-4dda-97aa-a1d14e648840 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:13.734206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:13.735640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550923ad-e041-4342-867a-0a551194178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:13.736320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550923ad-e041-4342-867a-0a551194178d None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.736607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550923ad-e041-4342-867a-0a551194178d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.736903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550923ad-e041-4342-867a-0a551194178d None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/105ad03a7a5c40c78500f0ee01583fae/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.738892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550923ad-e041-4342-867a-0a551194178d None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0ee695ae09442182b5b3e6795cbcc8, user_id=105ad03a7a5c40c78500f0ee01583fae, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:13.740413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550923ad-e041-4342-867a-0a551194178d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:13.806917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72d8f-cec4-4a88-a93f-98f0a5326123 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.807199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72d8f-cec4-4a88-a93f-98f0a5326123 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.807477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e72d8f-cec4-4a88-a93f-98f0a5326123 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.808999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 54/106] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:13.813924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb23747-ee88-4eac-99b5-2531e6525494 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.814208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb23747-ee88-4eac-99b5-2531e6525494 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.814480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb23747-ee88-4eac-99b5-2531e6525494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.826167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 53/107] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:13 2022] PUT /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/105ad03a7a5c40c78500f0ee01583fae/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:13.846084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9fb23747-ee88-4eac-99b5-2531e6525494 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:13.847300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9fb23747-ee88-4eac-99b5-2531e6525494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:13.871931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:13.935640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 55/108] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:13.963935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9263f-0c1d-40a1-a876-8d6559ad0317 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.964208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9263f-0c1d-40a1-a876-8d6559ad0317 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.964421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9263f-0c1d-40a1-a876-8d6559ad0317 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.965820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 54/109] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:13.971630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:13.971926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:13.972212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:13.997018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:13.998384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:14.087900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 56/110] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:14.102437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9263f-0c1d-40a1-a876-8d6559ad0317 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.222141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9263f-0c1d-40a1-a876-8d6559ad0317 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.257859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:14.258501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.258786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.259062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.270521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:14.276279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:14.277176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:14.277932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 55/111] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:14.283398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.375221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.384749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.404661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d84ff5-4f65-47dd-b1be-a2964093f1e5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.426383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.439456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:14.443707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.443707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.443707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.443707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:14.443707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:14.443707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9b94-d969-4c76-94e3-d273e74e900b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:14.452518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 57/112] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:14.456906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75e940-9381-4de6-893b-b6db9b8ab6b7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.456906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75e940-9381-4de6-893b-b6db9b8ab6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.456906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75e940-9381-4de6-893b-b6db9b8ab6b7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.456906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 58/113] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:14.461641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6996-6d75-466a-9ce7-4c3394a4dcd7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.461968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6996-6d75-466a-9ce7-4c3394a4dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.462215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc6996-6d75-466a-9ce7-4c3394a4dcd7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.463648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 59/114] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:14.464297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc6996-6d75-466a-9ce7-4c3394a4dcd7 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.491115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82dc4f-dc8e-431a-b3ba-81d4ff650e0c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.529017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a82c97-1519-4953-bffa-d446dafa9e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:14.529663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82c97-1519-4953-bffa-d446dafa9e25 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.529948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82c97-1519-4953-bffa-d446dafa9e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.530235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a82c97-1519-4953-bffa-d446dafa9e25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.542289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a82c97-1519-4953-bffa-d446dafa9e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-64a82c97-1519-4953-bffa-d446dafa9e25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:14.547600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:14.548657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:14.549502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 56/115] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:14.550212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dfe8d-1d61-4a4b-a65d-b4b520c2785d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.550505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dfe8d-1d61-4a4b-a65d-b4b520c2785d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.550843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dfe8d-1d61-4a4b-a65d-b4b520c2785d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.569279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ab9dfe8d-1d61-4a4b-a65d-b4b520c2785d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:14.570529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ab9dfe8d-1d61-4a4b-a65d-b4b520c2785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:14.579734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc6996-6d75-466a-9ce7-4c3394a4dcd7 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.622125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:14.622765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.623050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.623329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.633948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:14.638726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:14.639774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:14.640541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 60/116] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:14.641570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.645076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.654480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 57/117] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:14.658247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.658881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.659169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.683758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:14.685016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:14.725028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.752941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d1203-0c54-4802-971d-2b14164a2609 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.775418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 58/118] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:14.776983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.791713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:14.792718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.793273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.793771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.795216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:14.796915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:14.798816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 61/119] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:14.799832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:14.883395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.919915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77193fe-8dda-45bc-911f-2d8214545949 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.959268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:14.960405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.960922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.961480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.963900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:14.963900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:14.964387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:14.966179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf59435-8880-451a-8bae-4508852e03b1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:14.974636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 62/120] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:14.975864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.059052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.059843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.060199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.060550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.061351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:15.061726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.063209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.072099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 59/121] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.076762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.098960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e374-e322-4d60-be83-cf5444956243 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.106823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.137431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.138101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.138385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.138818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.139852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.141339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.147657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 63/122] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:14 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.147657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.147657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.187370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da85f5-95ed-48ba-9365-d8ba028c8610 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.223447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.224091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.224377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.224654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.226965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.228430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.230077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 60/123] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.234683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.252084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42fe9f-54e1-412a-902e-8aefa3bd7d53 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.287547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.288735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.289146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.289443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.300071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:15.305766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:15.306660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:15.307594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 64/124] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:15.310582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.337857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.344693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb64f9-e9d5-4880-aef0-de5286e35ef0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.373648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.382400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.383123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.383405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.383709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] PATH_INFO: `/v3/users/aeb16018c2bb4574b78efa30ae2c3dd0` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.399048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb16018c2bb4574b78efa30ae2c3dd0)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.401031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.403482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 61/125] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/users/aeb16018c2bb4574b78efa30ae2c3dd0 => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.404306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.415322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06a11-56ac-480c-b330-0053a55fdc06 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.457391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.458022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.458306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.458571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.459698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.461844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.463395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 65/126] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.464208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.516401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82bd6-aed6-491d-b794-2cf7508f6764 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.551739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.552554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.552836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.553432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/135d762e2a5b413aa4e27f4cc35ec7e6/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.556762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0ee695ae09442182b5b3e6795cbcc8, user_id=135d762e2a5b413aa4e27f4cc35ec7e6, role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.558521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.573673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d608d0-7baf-4d1d-b7a3-c204d0e23373 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.596813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.603016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.603640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.603919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.604195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.605325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.606785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.613577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 66/127] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.613577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.630965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 62/128] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:15 2022] PUT /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/135d762e2a5b413aa4e27f4cc35ec7e6/roles/97b084644adb4949ab768912c0bb75e8 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:15.633708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.682262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.729581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb276f0-9e3f-4a1c-9af7-11333503f88c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.770123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.770918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.771399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.771718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.773150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:15.773983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.775566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.785323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 67/129] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:15.786351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:15.824188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52881ae6-4ff2-4347-bca5-495a35abcca4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.859703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:15.876788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.877443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.881119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.881629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.909781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d2bb1-7059-4897-bd34-3baefc87edbb None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:15.955280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:15.955955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:15.956247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:15.956526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:15.957744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.960105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:15.987783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:15.989427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:16.005528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:16.053124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 68/130] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:14:15 2022] POST /identity/v3/projects => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:16.056611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:16.139323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 63/131] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:16.141148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:16.277672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e95b5-d9c1-4b7b-a1b0-4c8a7684d5c2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:16.295809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:16.318184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96375e-77a1-4d26-a652-252a69a55d21 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:16.322389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:16.323046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.323328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.323605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.325158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:16.326754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:16.329033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 64/132] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:16 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:16.329816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:16.335571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:16.363036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ea255-85c4-42b6-a006-64b0af7b4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:16.364082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea255-85c4-42b6-a006-64b0af7b4b78 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.364442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea255-85c4-42b6-a006-64b0af7b4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.365542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ea255-85c4-42b6-a006-64b0af7b4b78 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/135d762e2a5b413aa4e27f4cc35ec7e6/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.367512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea255-85c4-42b6-a006-64b0af7b4b78 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0ee695ae09442182b5b3e6795cbcc8, user_id=135d762e2a5b413aa4e27f4cc35ec7e6, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:16.369528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ea255-85c4-42b6-a006-64b0af7b4b78 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:16.443658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 69/133] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:16 2022] PUT /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/135d762e2a5b413aa4e27f4cc35ec7e6/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:16.447680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01279732-8ce4-4878-b0df-0c0297661068 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.447680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01279732-8ce4-4878-b0df-0c0297661068 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.447680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01279732-8ce4-4878-b0df-0c0297661068 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.448259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 70/134] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:16.449127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01279732-8ce4-4878-b0df-0c0297661068 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:16.450671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df7c01-985b-4bbc-88ad-a6aad1803c9d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:16.525543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6170f6-de14-443a-99cc-a53c6654f77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:16.526434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6170f6-de14-443a-99cc-a53c6654f77e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.526980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6170f6-de14-443a-99cc-a53c6654f77e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.527398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6170f6-de14-443a-99cc-a53c6654f77e None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.531091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6170f6-de14-443a-99cc-a53c6654f77e None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:16.532792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6170f6-de14-443a-99cc-a53c6654f77e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:16.534933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 65/135] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:16 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:16.536243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.536676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.537125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.578082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:16.579590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:16.584062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01279732-8ce4-4878-b0df-0c0297661068 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:16.627966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:16.629027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.629416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.629814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] PATH_INFO: `/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.631281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] RBAC: Authorizing `identity:get_project(project_id=c0d011eb1d8d4352b06cfda0ed150bb9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:16.633755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:16.635876 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 71/136] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:16 2022] GET /identity/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:16.636994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:16.683961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:16.699115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:16.734218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 66/137] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:16.735100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:16.780851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013eb0f8-aaf3-4239-9a83-3040b6a6e09b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:16.820593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:16.821033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.821582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.822217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.823389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] RBAC: Adding query filter params (role.id=176951ed62024e8493f28661d97c0cdd, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=aeb16018c2bb4574b78efa30ae2c3dd0) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:16.824782 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:16.826523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:16.832445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 72/138] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:16 2022] GET /identity/v3/role_assignments?user.id=aeb16018c2bb4574b78efa30ae2c3dd0&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=176951ed62024e8493f28661d97c0cdd => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:16.928218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f64b8-ae35-47ac-9c5e-3f698f72a191 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:16.976753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:16.977775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:16.978221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:16.978660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] PATH_INFO: `/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9/users/135d762e2a5b413aa4e27f4cc35ec7e6/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:16.982334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] RBAC: Authorizing `identity:create_grant(project_id=c0d011eb1d8d4352b06cfda0ed150bb9, user_id=135d762e2a5b413aa4e27f4cc35ec7e6, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:16.983823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:17.057752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 67/139] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:16 2022] PUT /identity/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9/users/135d762e2a5b413aa4e27f4cc35ec7e6/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:17.174292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:17.310420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcea22-c6c8-44b8-8128-f630d97e3c0b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:17.312754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:17.354961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:17.355619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.355863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.356139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:17.371741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:17.376582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:17.377516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:17.378406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 73/140] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:17.386056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:17.465231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:17.537569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e332bc-2e63-4ea8-8202-010c12bb2b5f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:17.581970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:17.582518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.582807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.583088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:17.583767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:17.584076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:17.585602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:17.594252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 68/141] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:17.605979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:17.764670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7076ab-096e-4eec-80e1-38c0749cb3aa None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:17.810497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:17.811140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.811420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.811694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] PATH_INFO: `/v3/users/105ad03a7a5c40c78500f0ee01583fae` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:17.812885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] RBAC: Authorizing `identity:get_user(user_id=105ad03a7a5c40c78500f0ee01583fae)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:17.814370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:17.815966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 74/142] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:17 2022] GET /identity/v3/users/105ad03a7a5c40c78500f0ee01583fae => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:17.822047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:17.937650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a7c50-df1c-4f14-a564-085a4a5211b8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:17.939132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:17.984303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:17.984951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.985242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.985522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:18.059542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:18.060989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:18.169607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:18.172343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 69/143] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:18.220396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:18.306923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595ba22-2155-4aa2-ad5b-e7a4540a0bae None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:18.343269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56d11c-214f-4003-a5e0-ce9cc9b38d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:18.344252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56d11c-214f-4003-a5e0-ce9cc9b38d12 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:18.344743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56d11c-214f-4003-a5e0-ce9cc9b38d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:18.345246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56d11c-214f-4003-a5e0-ce9cc9b38d12 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:18.346560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56d11c-214f-4003-a5e0-ce9cc9b38d12 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:18.348187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56d11c-214f-4003-a5e0-ce9cc9b38d12 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:18.350353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 75/144] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:18 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:18.355968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:18.361232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:18.421540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f3e7f-f68f-4d71-9ec2-93917a6fc378 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:18.422501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f3e7f-f68f-4d71-9ec2-93917a6fc378 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:18.423034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f3e7f-f68f-4d71-9ec2-93917a6fc378 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:18.424629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 76/145] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:18.429832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:18.430413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:18.430890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:18.450488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:18.452275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:18.484735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15b96a-b8e5-4520-af32-a3f93bf89c8b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:18.525268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:18.525894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:18.526178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:18.526458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:18.527198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=105ad03a7a5c40c78500f0ee01583fae) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:18.527513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:18.532914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:18.535808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 70/146] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:18 2022] GET /identity/v3/role_assignments?user.id=105ad03a7a5c40c78500f0ee01583fae&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:18.576425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 77/147] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:18.977959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:19.061546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:19.105482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8742991-f7a8-4c05-89df-efd45ce007a2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:19.146568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84732d-b06f-4564-9e9b-38f1f3a4c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:19.147625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84732d-b06f-4564-9e9b-38f1f3a4c05a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.148169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84732d-b06f-4564-9e9b-38f1f3a4c05a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.148645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84732d-b06f-4564-9e9b-38f1f3a4c05a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.160878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a84732d-b06f-4564-9e9b-38f1f3a4c05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1a84732d-b06f-4564-9e9b-38f1f3a4c05a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:19.167233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:19.168107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:19.168894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:19.171154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 71/148] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:19.177142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:19.215200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:19.318867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4750c-14bb-40fb-bdf0-3082b9a407d0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:19.341154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc65d55-3a96-4597-b8a5-5c56f792612a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.341451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc65d55-3a96-4597-b8a5-5c56f792612a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.341726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc65d55-3a96-4597-b8a5-5c56f792612a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.343216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 72/149] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:19.350071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.350353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.350730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.373508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:19.374699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:19.388365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:19.389024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.389307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.389584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.390262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:19.390564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:19.391947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:19.407816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 78/150] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:19.414106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:19.452988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 73/151] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:19.573297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:19.579129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae91d024-1c25-466d-a5eb-2417292d624b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:19.639253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701612f3-aca0-4f1b-8110-4b845ba0df88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:19.639914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701612f3-aca0-4f1b-8110-4b845ba0df88 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.640196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701612f3-aca0-4f1b-8110-4b845ba0df88 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.640475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701612f3-aca0-4f1b-8110-4b845ba0df88 None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.641715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701612f3-aca0-4f1b-8110-4b845ba0df88 None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:19.643213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701612f3-aca0-4f1b-8110-4b845ba0df88 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:19.644837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 79/152] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:19 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:19.651280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:19.725140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:19.743854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483893c-b5eb-4ae2-ada4-57819f231475 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.744172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483893c-b5eb-4ae2-ada4-57819f231475 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.744402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483893c-b5eb-4ae2-ada4-57819f231475 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.745777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 80/153] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:19.751014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.751399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.751773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.772106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c2527-e2eb-4845-9688-8c48b89d0b13 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:19.786247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:19.787449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:19.808136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:19.808767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.809080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.809414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.898728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:19.900148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:19.913216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 81/154] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:19.914297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:19.959242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:20.025147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 74/155] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:20.030746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.061256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:20.086817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bbfd5-39d6-4fa7-9c83-e4c7a17e3651 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.134160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85d16-9f91-4c8d-9ac3-d955018802ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.135180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85d16-9f91-4c8d-9ac3-d955018802ac None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.135663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85d16-9f91-4c8d-9ac3-d955018802ac None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.136146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85d16-9f91-4c8d-9ac3-d955018802ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.138461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb85d16-9f91-4c8d-9ac3-d955018802ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:20.140153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb85d16-9f91-4c8d-9ac3-d955018802ac None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:20.142807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 82/156] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:20.144664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6df4b-b0ec-43e3-9db8-81ac78b7ad9a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.145193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6df4b-b0ec-43e3-9db8-81ac78b7ad9a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.146117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6df4b-b0ec-43e3-9db8-81ac78b7ad9a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.148496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 83/157] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:20.154233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.154745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.155222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.174131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e059fff-3255-466d-97b8-508cdc74cd80 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.174693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:20.176101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:20.211180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.211875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.212345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.212799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.214141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:20.215933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:20.218339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 75/158] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:20.219405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.245857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:20.259959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 84/159] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:20.261034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.329617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07953269-1892-4ff8-aa26-81ece78517e6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.379142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:20.384558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.385432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.385870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.386293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.387565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:20.389498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:20.394338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba198b3-8486-4494-b6c9-264cf5662878 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.439181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.439775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.440059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.440343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.441229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] RBAC: Adding query filter params (role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=135d762e2a5b413aa4e27f4cc35ec7e6) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:20.441546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:20.443055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:20.450420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 85/160] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/role_assignments?user.id=135d762e2a5b413aa4e27f4cc35ec7e6&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=97b084644adb4949ab768912c0bb75e8 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:20.451058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.509458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 76/161] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:20 2022] POST /identity/v3/users => generated 283 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:20.575153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.583208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e51ee-b4d3-4540-accc-0d4ee2672c93 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.629318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.629983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.631254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.631536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.655625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:20.661306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:20.662223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:20.663697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 86/162] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:20.669866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.688739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:20.739439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:20.764830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0895e-ceee-4876-b463-035d856751e3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.794999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b6bcd-d567-49a9-908e-1460d8e47a26 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.805419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69906c24-8518-479e-9b13-3dc48193a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.806062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906c24-8518-479e-9b13-3dc48193a787 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.806443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906c24-8518-479e-9b13-3dc48193a787 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.806718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906c24-8518-479e-9b13-3dc48193a787 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.817773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69906c24-8518-479e-9b13-3dc48193a787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-69906c24-8518-479e-9b13-3dc48193a787 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:20.823622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:20.824527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:20.825359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 77/163] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:20.826240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf1ea-c817-47d9-8842-fddd1e7014fd None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.826531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf1ea-c817-47d9-8842-fddd1e7014fd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.827567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf1ea-c817-47d9-8842-fddd1e7014fd None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.828710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 78/164] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:20.832940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf1ea-c817-47d9-8842-fddd1e7014fd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.840657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:20.841303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.841582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.841849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.842511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:20.843259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:20.844648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5539d5-5a7b-4507-8473-d44a6bab0326 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:20.856982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 87/165] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:20.858201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:20.858495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:20.858791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:20.879228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:20.880437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:20.966638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf1ea-c817-47d9-8842-fddd1e7014fd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:20.980731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 88/166] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:20.981556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:20.984903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.011689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.012377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.012672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.012973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.013834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:21.014484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.016190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.027982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 79/167] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:21.034094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.037340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.106052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab416b6-f2a7-4c24-b950-ec7a1f3d0113 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.144586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.145419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.145801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.146180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.147464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.149419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.151184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 89/168] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:20 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:21.157040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.163050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b197eea-e32a-41d3-9f7a-f8a075e752d2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.203344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697984-b011-440b-9838-c969e89989fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.203986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697984-b011-440b-9838-c969e89989fa None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.204295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697984-b011-440b-9838-c969e89989fa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.205084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697984-b011-440b-9838-c969e89989fa None admin] PATH_INFO: `/v3/users/aeb16018c2bb4574b78efa30ae2c3dd0` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.206249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d697984-b011-440b-9838-c969e89989fa None admin] RBAC: Authorizing `identity:get_user(user_id=aeb16018c2bb4574b78efa30ae2c3dd0)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.208157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d697984-b011-440b-9838-c969e89989fa None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.210588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 80/169] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/users/aeb16018c2bb4574b78efa30ae2c3dd0 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:21.216245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697984-b011-440b-9838-c969e89989fa None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.241565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.278050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec49e75-e49d-4872-adba-82f678976d80 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.299048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.319887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.320512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.320788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.321894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.335670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697984-b011-440b-9838-c969e89989fa None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.394836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.395594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.396032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.396318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.398328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.399813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.401965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 81/170] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:21.403131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.405713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.409690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.509913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.535153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016bbd3-39d3-47cb-870e-d1bef6c55bb0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.555266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 90/171] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:21.555266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.580762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.581890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.582494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.582996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.591766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.599491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:21.605724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:21.606908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:21.607710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:21.610099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 82/172] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:21.611148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.682022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fbc49-84ed-4b31-929d-08220e3310cd None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.747821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0605d90-88f6-4bf1-865b-07903e951b2f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.784133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.784780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.785115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.785447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.786695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.788833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.790945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 83/173] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:21.791551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.817606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.818294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.818576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.818850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/aeb16018c2bb4574b78efa30ae2c3dd0/roles/176951ed62024e8493f28661d97c0cdd` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.825618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0ee695ae09442182b5b3e6795cbcc8, user_id=aeb16018c2bb4574b78efa30ae2c3dd0, role_id=176951ed62024e8493f28661d97c0cdd)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.827083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.862992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.899212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f8f89-341a-441a-a18d-ee4d996cbc20 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.931305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 91/174] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:21 2022] PUT /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/users/aeb16018c2bb4574b78efa30ae2c3dd0/roles/176951ed62024e8493f28661d97c0cdd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:21.932423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.943735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.943735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.943735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.943735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.944282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:21.944282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.945441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.970158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 84/175] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:21.977727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:22.015689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:22.146889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872de5f3-eded-4cfb-a59d-1040460eae77 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:22.204643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f3ca6-0291-4fee-8834-564a6de6161c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:22.220093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:22.220709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:22.221009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:22.221287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:22.222012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=135d762e2a5b413aa4e27f4cc35ec7e6) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:22.222314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:22.223761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:22.238614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 92/176] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/role_assignments?user.id=135d762e2a5b413aa4e27f4cc35ec7e6&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=76c32cb614c145319ad750282d6d3551 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:22.256225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:22.265842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:22.266489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:22.270074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:22.270074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] PATH_INFO: `/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:22.270074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] RBAC: Authorizing `identity:get_user(user_id=135d762e2a5b413aa4e27f4cc35ec7e6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:22.271232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:22.273136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 85/177] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/users/135d762e2a5b413aa4e27f4cc35ec7e6 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:22.279090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:22.424204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:22.430625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a3a92-b76a-4e1c-adf2-4a4d83c85cc6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:22.477682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:22.478326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:22.478610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:22.478888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:22.567812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:22.569429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:22.712423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:22.720748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 93/178] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:22.727606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:22.843807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291356-b062-4515-a5d7-786da9add4ba None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:22.885573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2656d3-2ded-41ac-9daf-0cf355e7056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:22.886218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2656d3-2ded-41ac-9daf-0cf355e7056d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:22.886497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2656d3-2ded-41ac-9daf-0cf355e7056d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:22.886769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2656d3-2ded-41ac-9daf-0cf355e7056d None admin] PATH_INFO: `/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:22.887943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2656d3-2ded-41ac-9daf-0cf355e7056d None admin] RBAC: Authorizing `identity:get_project(project_id=c0d011eb1d8d4352b06cfda0ed150bb9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:22.889420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2656d3-2ded-41ac-9daf-0cf355e7056d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:22.891359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 86/179] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:22 2022] GET /identity/v3/projects/c0d011eb1d8d4352b06cfda0ed150bb9 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:22.896926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:23.030403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0e51e-1389-4510-863b-d3d7997b0894 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:23.075402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:23.076386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:23.076873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:23.077363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:23.078280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=c0d011eb1d8d4352b06cfda0ed150bb9, user.id=135d762e2a5b413aa4e27f4cc35ec7e6) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:23.078806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:23.080436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b74df7-257f-4557-a9dc-14cc1438b18b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:23.089717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 94/180] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:22 2022] GET /identity/v3/role_assignments?user.id=135d762e2a5b413aa4e27f4cc35ec7e6&scope.project.id=c0d011eb1d8d4352b06cfda0ed150bb9&role.id=76c32cb614c145319ad750282d6d3551 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:23.517049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76623382-39bc-49a6-8dcd-3c477291cc06 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:23.517412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76623382-39bc-49a6-8dcd-3c477291cc06 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:23.517611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76623382-39bc-49a6-8dcd-3c477291cc06 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:23.518948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 87/181] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:23.524966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:23.525251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:23.525521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:23.547190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:23.548369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:23.618209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:23.629536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 95/182] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:23.920366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdcc03-515a-4ced-a2d8-766bc5e26d2a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:23.920897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdcc03-515a-4ced-a2d8-766bc5e26d2a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:23.920897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdcc03-515a-4ced-a2d8-766bc5e26d2a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:23.922470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 88/183] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:23.925332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:23.927603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:23.927790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:23.928018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:23.949162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:23.950375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:24.015578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.025654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 89/184] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:24.048528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2b49a-b3a5-4f9f-86b1-456fdc5bfd33 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.085292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c1ba0-e5c8-4c1b-89aa-9ba941dfb25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.086374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c1ba0-e5c8-4c1b-89aa-9ba941dfb25b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.086854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c1ba0-e5c8-4c1b-89aa-9ba941dfb25b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.087298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c1ba0-e5c8-4c1b-89aa-9ba941dfb25b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.098081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c1ba0-e5c8-4c1b-89aa-9ba941dfb25b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-9e6c1ba0-e5c8-4c1b-89aa-9ba941dfb25b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:24.104244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:24.105211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:24.105980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:24.107932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 96/185] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:24.114895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.150853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139e5d5-c23a-4b90-af13-c98d2e2ff61d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.151392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139e5d5-c23a-4b90-af13-c98d2e2ff61d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.151808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139e5d5-c23a-4b90-af13-c98d2e2ff61d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.154307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 97/186] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:24.164796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.165359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.165828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.189489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:24.191577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:24.232434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d23e5-4d29-4226-be93-de1281562a97 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.268516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.269246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.269351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.269822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.269926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.270670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:24.271036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.272729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:24.281599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 90/187] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:24.282825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 98/188] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:24.291642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.298896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.362851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.408490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8aff96-54fc-4483-a609-292378630dee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.424490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca518524-c7bf-4d49-b5f7-b93beb1dcb5c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.446217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.447071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.447347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.447623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] PATH_INFO: `/v3/users/8db0453327314209918d011876fbd7d4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.464901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] RBAC: Authorizing `identity:get_user(user_id=8db0453327314209918d011876fbd7d4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.466379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:24.468452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 91/189] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/users/8db0453327314209918d011876fbd7d4 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:24.473740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.476923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.478741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.479231 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.479708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.493925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:24.500534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:24.501455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:24.502264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:24.504995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 99/190] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:24.508450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.559218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.605507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a90147-09d3-4891-801a-a884085e03bb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.622614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.628665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab0847-e2a4-4899-8f60-c11c51bdef73 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.646424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f912d-b259-4ebf-8548-cde30ac31be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.647055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f912d-b259-4ebf-8548-cde30ac31be6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.647872 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f912d-b259-4ebf-8548-cde30ac31be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.648615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f912d-b259-4ebf-8548-cde30ac31be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.667604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.668211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.668484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.668753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.669474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:24.669749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.671121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:24.679226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 100/191] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:24.681877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.726329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f912d-b259-4ebf-8548-cde30ac31be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.727858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f912d-b259-4ebf-8548-cde30ac31be6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:24.786325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2befbb5-2d23-4025-95b7-b9d97551650b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.787912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.819781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.820440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.821597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.821871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.823002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.824439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:24.826013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 101/192] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:24.826764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.835061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 92/193] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:24.836581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37ff26-8b12-40a7-9689-92d9e89f760a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.836892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37ff26-8b12-40a7-9689-92d9e89f760a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.837164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37ff26-8b12-40a7-9689-92d9e89f760a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.839464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 93/194] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:24.839464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37ff26-8b12-40a7-9689-92d9e89f760a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.855046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:24.926596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591791c3-a961-4675-b2f6-b31d92e9b474 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.942566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37ff26-8b12-40a7-9689-92d9e89f760a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:24.960547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.961196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.961474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.961744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] PATH_INFO: `/v3/users/8db0453327314209918d011876fbd7d4` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.962858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] RBAC: Authorizing `identity:get_user(user_id=8db0453327314209918d011876fbd7d4)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.964293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:24.970416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 102/195] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/users/8db0453327314209918d011876fbd7d4 => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:24.970416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:24.977359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0cab4-3f14-48df-b8a2-11f8b285afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:24.977909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0cab4-3f14-48df-b8a2-11f8b285afbc None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.978189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0cab4-3f14-48df-b8a2-11f8b285afbc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.978464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0cab4-3f14-48df-b8a2-11f8b285afbc None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.979106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0cab4-3f14-48df-b8a2-11f8b285afbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:24.980545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0cab4-3f14-48df-b8a2-11f8b285afbc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.008853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.066253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0704-2872-4c26-b710-2fccf9d7bf4c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.086934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 94/196] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:24 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:25.088429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.089019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.089324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.109628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:25.110867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:25.111988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.112650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.113024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.113309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.114601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:25.116735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.123887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 103/197] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:24 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:25.123887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.198544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.203434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 95/198] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:25.204270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.237190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788483fb-4eba-49c6-8a08-287c03f1268b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.247738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.285262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.286206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.286490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.286908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.350003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096809e-54a8-4bc0-bb3e-9699c657f3a8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.376264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:25.379214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.396101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.396735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.397796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.398283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.399455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=8db0453327314209918d011876fbd7d4) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:25.400061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:25.401708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.409305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 96/199] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/role_assignments?user.id=8db0453327314209918d011876fbd7d4&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:25.502312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.509855 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 104/200] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:25.521772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.531944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.601485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.654217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4604ba-90b0-4576-975e-d1105d345563 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.655432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a47f-609d-42d6-99bd-d141fbbc3ce1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.693724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.695215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.695729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.696232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.698024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:25.699751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.701727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 105/201] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:25.712340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.723798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.724733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.725934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.726405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.737849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:25.745355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:25.746253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:25.747051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:25.749726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 97/202] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:25.755516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.871057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.921694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7c825-51ac-41a6-b050-d2257d4c7c4d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.936188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb62988-107c-498f-83b2-419ddbf0ddfa None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.977627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.979180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.979809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.980303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.981302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] RBAC: Adding query filter params (role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=8db0453327314209918d011876fbd7d4) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:25.982438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.982861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:25.985057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.998308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 106/203] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/role_assignments?user.id=8db0453327314209918d011876fbd7d4&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=97b084644adb4949ab768912c0bb75e8 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:26.007911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:26.008559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:26.008837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:26.009135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:26.009834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:26.010134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:26.011577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:26.031732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 98/204] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:26.037386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:26.191033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18040657-b420-4f48-80c2-d3a3d8902486 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:26.233992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:26.234641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:26.234922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:26.235195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] PATH_INFO: `/v3/users/aeb16018c2bb4574b78efa30ae2c3dd0` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:26.236354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] RBAC: Authorizing `identity:get_user(user_id=aeb16018c2bb4574b78efa30ae2c3dd0)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:26.237865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:26.239522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 107/205] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:26 2022] GET /identity/v3/users/aeb16018c2bb4574b78efa30ae2c3dd0 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:26.246042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:26.362411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cca9e6-0159-4a86-98c6-d5db880b779a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:26.406591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:26.407206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:26.407481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:26.407748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:26.480842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:26.494919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:26.496715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:26.618985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 99/206] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:26.625490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:26.646975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:26.732265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ea0dc-111f-47ac-bc35-3b7567e376d7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:26.772141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a410f20-8c76-4b62-80db-985b0bf20803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:26.773000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a410f20-8c76-4b62-80db-985b0bf20803 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:26.773281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a410f20-8c76-4b62-80db-985b0bf20803 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:26.773556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a410f20-8c76-4b62-80db-985b0bf20803 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:26.774679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a410f20-8c76-4b62-80db-985b0bf20803 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:26.776182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a410f20-8c76-4b62-80db-985b0bf20803 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:26.777834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 108/207] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:26 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:26.783925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:26.914023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:26.933990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49452e-6dc1-4e3a-8dde-aaf59637749f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:27.023918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:27.024545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:27.024805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:27.025131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:27.025864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] RBAC: Adding query filter params (role.id=176951ed62024e8493f28661d97c0cdd, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8, user.id=aeb16018c2bb4574b78efa30ae2c3dd0) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:27.026178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:27.027630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353dc2a-f128-4c24-86f6-071bd1661f1f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:27.041195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 100/208] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:26 2022] GET /identity/v3/role_assignments?user.id=aeb16018c2bb4574b78efa30ae2c3dd0&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=176951ed62024e8493f28661d97c0cdd => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:27.945980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a1d8-c7a3-4236-a886-d94bcdef4b18 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:27.946778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a1d8-c7a3-4236-a886-d94bcdef4b18 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:27.946778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4a1d8-c7a3-4236-a886-d94bcdef4b18 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:27.948539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 109/209] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:27.954663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b867c8-1fc8-4c80-845f-129d0d7d04be None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:27.954945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b867c8-1fc8-4c80-845f-129d0d7d04be None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:27.955212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b867c8-1fc8-4c80-845f-129d0d7d04be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.000729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e836-46a9-4a5e-93cd-141b81899212 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.001059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e836-46a9-4a5e-93cd-141b81899212 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.001334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4e836-46a9-4a5e-93cd-141b81899212 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.002857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 110/210] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:28.007884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.008143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.008426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.023188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-44b867c8-1fc8-4c80-845f-129d0d7d04be None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:28.024725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-44b867c8-1fc8-4c80-845f-129d0d7d04be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.028008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:28.029443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.108539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 101/212] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:28.108539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2168d-42b5-466b-ae4e-021203890ff2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.108539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2168d-42b5-466b-ae4e-021203890ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.108539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2168d-42b5-466b-ae4e-021203890ff2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.108539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 102/213] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:28.109336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 111/213] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:28.112526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.113040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.113521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:28.134467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.205874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 103/214] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:28.382734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.385991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.387152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.401834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.503870 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d534137-8e3f-49bb-b453-be4e89ab18a8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:28.514664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b4861-bad6-4371-9fb7-9aab1f781194 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:28.542012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:28.542649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.542928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.543200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.544306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:28.545810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:28.547743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 112/215] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:28.548401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.558584 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:28.559397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.559676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.559975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.571184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:28.576757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:28.577657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:28.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 104/216] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:28.579108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.630167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.660786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.661106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474620cf-e315-42c4-88af-2119e59ed11d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:28.691643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3529-06e8-42f4-bdbc-ebcd8a534000 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:28.695538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:28.696158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.696682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.696981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.707074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:28.712988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:28.713984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:28.714777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 113/217] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:28.716302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.728358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f52161-df85-4fcf-85fe-cca8445890d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:28.728723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f52161-df85-4fcf-85fe-cca8445890d2 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.729030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f52161-df85-4fcf-85fe-cca8445890d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.729301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f52161-df85-4fcf-85fe-cca8445890d2 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.729943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f52161-df85-4fcf-85fe-cca8445890d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:28.731370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f52161-df85-4fcf-85fe-cca8445890d2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:28.813900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7a4ff-cd23-41d9-b826-d34510e49c24 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:28.842014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.846841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:28.847445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.847730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.848005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.848695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:28.849036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:28.851435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861598b-5c9c-4b9c-95e2-bf12beb57c10 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:28.853176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 105/218] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:28 2022] POST /identity/v3/users => generated 297 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:28.854404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d43b-8cea-4c0f-85a0-222c72ac8e83 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.854698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d43b-8cea-4c0f-85a0-222c72ac8e83 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.860548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249d43b-8cea-4c0f-85a0-222c72ac8e83 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.860548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 106/219] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:28.860548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d43b-8cea-4c0f-85a0-222c72ac8e83 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.866436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 114/220] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:28.867582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.867867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.868140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.888190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:28.889460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.972480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 115/221] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:28.974240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.015790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249d43b-8cea-4c0f-85a0-222c72ac8e83 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.049972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:29.065112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.066142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.066436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.066711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.067393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:29.067699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.071653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.081577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 107/222] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:29.087610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.135573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4d539-64ad-4d3a-8f79-71e63e54d1a2 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.170854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:29.188406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313697d-9162-456f-87ad-c27822171130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.188406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313697d-9162-456f-87ad-c27822171130 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.188406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313697d-9162-456f-87ad-c27822171130 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.188743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313697d-9162-456f-87ad-c27822171130 None admin] PATH_INFO: `/v3/users/04866f6f226a40bc9a7a67f5f5c9dbaa` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.208833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313697d-9162-456f-87ad-c27822171130 None admin] RBAC: Authorizing `identity:get_user(user_id=04866f6f226a40bc9a7a67f5f5c9dbaa)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.211489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313697d-9162-456f-87ad-c27822171130 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.213518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 116/223] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/users/04866f6f226a40bc9a7a67f5f5c9dbaa => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:29.218687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540adb4-217b-45c9-90cf-362af91a9c6a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.219674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313697d-9162-456f-87ad-c27822171130 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.266392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.267054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.269340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.269857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] PATH_INFO: `/v3/users/8db0453327314209918d011876fbd7d4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.271449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] RBAC: Authorizing `identity:get_user(user_id=8db0453327314209918d011876fbd7d4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.273702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.276454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 108/224] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/users/8db0453327314209918d011876fbd7d4 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:29.284266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.353595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313697d-9162-456f-87ad-c27822171130 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.361949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:29.404649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.405293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.405570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.405842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.423176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebaaf6-49e4-4cd6-9aea-77eec7845305 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.466093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.466729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.467003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.467277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.468500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.470093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.472283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 109/225] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:29.478234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:29.479290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.498096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.500201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.614159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ba6b-ba49-49a4-9285-9cc4c55df80f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.628383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 117/226] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:29.629256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.647368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:29.664048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.664987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.665645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.667028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.681306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:29.688486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:29.689417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:29.690242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 110/227] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:29.690916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.719757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:29.738660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72eef7-577a-4c4a-8a5f-d6b39bc7b140 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.777487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.778332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.778619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.778896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/8db0453327314209918d011876fbd7d4/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.781298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac8746c7d5e4606add57d76fb27178b, user_id=8db0453327314209918d011876fbd7d4, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.782855 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.822436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68a05a-3b33-4675-8741-2b0f848289bc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:29.857017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 118/228] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:29 2022] PUT /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/8db0453327314209918d011876fbd7d4/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:29.861786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.880063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:29.880726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:29.880998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:29.881306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:29.884202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:29.885746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:29.887862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 111/229] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:29.895714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:29.998636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:30.026692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184f5ec-8607-43bd-899f-f0b159de7d48 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:30.063164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5e1fd-5502-461a-b81d-2ec48296be00 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:30.106835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:30.107830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.108506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.109043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.109995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=04866f6f226a40bc9a7a67f5f5c9dbaa) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:30.110520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:30.112330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:30.119819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 112/230] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/role_assignments?user.id=04866f6f226a40bc9a7a67f5f5c9dbaa&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=76c32cb614c145319ad750282d6d3551 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:30.125878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:30.126531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.126814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.127093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.127791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:30.128194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:30.129882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:30.142104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 119/231] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:30.147935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:30.270778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095137c-9406-4803-a168-f2a3c965ed12 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:30.288672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:30.316408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:30.317088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.317434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.317709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] PATH_INFO: `/v3/users/8db0453327314209918d011876fbd7d4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.319394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] RBAC: Authorizing `identity:get_user(user_id=8db0453327314209918d011876fbd7d4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:30.320990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:30.326785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 113/232] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:30 2022] GET /identity/v3/users/8db0453327314209918d011876fbd7d4 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:30.332482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:30.340662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:30.491196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040de2f-15e2-4098-bc92-93789321dbe7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:30.527500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d186b7-9930-4f38-be63-f081517b0919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:30.528519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d186b7-9930-4f38-be63-f081517b0919 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.529136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d186b7-9930-4f38-be63-f081517b0919 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.529640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d186b7-9930-4f38-be63-f081517b0919 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.530980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d186b7-9930-4f38-be63-f081517b0919 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:30.532633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d186b7-9930-4f38-be63-f081517b0919 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:30.534806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 120/233] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:30 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:30.541127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:30.695178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16c6de-88fa-4f92-b22f-34e0680a1190 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:30.748041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc2b96-5642-4a15-8a6b-41bd3c8b605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:30.748634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc2b96-5642-4a15-8a6b-41bd3c8b605b None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.748941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc2b96-5642-4a15-8a6b-41bd3c8b605b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.749223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc2b96-5642-4a15-8a6b-41bd3c8b605b None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/8db0453327314209918d011876fbd7d4/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.751429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc2b96-5642-4a15-8a6b-41bd3c8b605b None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac8746c7d5e4606add57d76fb27178b, user_id=8db0453327314209918d011876fbd7d4, role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:30.752914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc2b96-5642-4a15-8a6b-41bd3c8b605b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:30.846252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 114/234] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:30 2022] PUT /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/8db0453327314209918d011876fbd7d4/roles/97b084644adb4949ab768912c0bb75e8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:31.685636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5f24-3fdb-487f-9e46-9b1725b52c74 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:31.686655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5f24-3fdb-487f-9e46-9b1725b52c74 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:31.687151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c5f24-3fdb-487f-9e46-9b1725b52c74 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:31.688979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 121/235] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:31.695422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:31.696597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:31.697095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:31.706189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4957dfa-d03c-436c-918b-1b406f592f7a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:31.706834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4957dfa-d03c-436c-918b-1b406f592f7a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:31.707308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4957dfa-d03c-436c-918b-1b406f592f7a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:31.708849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 122/236] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:31.722516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:31.723118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:31.723585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:31.725323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:31.726801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:31.758683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:31.760928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:31.811486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:31.843140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 115/237] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:31.898232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 123/238] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 04 10:14:32.125652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.250815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe499492-ca06-45d3-85cf-8c5fc75bace5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:32.298562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:32.299554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.300023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.300509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.315605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:32.320850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:32.322009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:32.322794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:32.325428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 116/239] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:32.331511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.334127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:32.434720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.469450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bcb7a-42f5-4d1f-baef-c6e7df787de5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:32.510574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:32.537153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:32.538336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.538612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.538883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.539557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:32.539857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:32.541275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:32.559285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 124/240] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:32.566630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.620806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bff5c-ede7-4de9-b69e-b1f4f8cac2e5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:32.633191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:32.667091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18468a46-0b5f-4a31-80f7-d7b558be8434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:32.668191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468a46-0b5f-4a31-80f7-d7b558be8434 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.668978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468a46-0b5f-4a31-80f7-d7b558be8434 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.669390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18468a46-0b5f-4a31-80f7-d7b558be8434 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.671271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18468a46-0b5f-4a31-80f7-d7b558be8434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:32.672491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9532495-8d16-4e99-bf96-3f1e4d97b820 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:32.673082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18468a46-0b5f-4a31-80f7-d7b558be8434 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:32.675918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 117/241] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:32.681696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d18c-85d4-4f29-be72-3c81411f99f3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.681922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d18c-85d4-4f29-be72-3c81411f99f3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.682531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88d18c-85d4-4f29-be72-3c81411f99f3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.683977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 118/242] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:32.684542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88d18c-85d4-4f29-be72-3c81411f99f3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.720168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f035-d5e8-47e5-8428-87fd8a72363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:32.721601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f035-d5e8-47e5-8428-87fd8a72363c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.722223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f035-d5e8-47e5-8428-87fd8a72363c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.722584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f035-d5e8-47e5-8428-87fd8a72363c None admin] PATH_INFO: `/v3/users/a341539db327425a8c3affa1e97f9c6f` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.740383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f035-d5e8-47e5-8428-87fd8a72363c None admin] RBAC: Authorizing `identity:get_user(user_id=a341539db327425a8c3affa1e97f9c6f)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:32.742642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f035-d5e8-47e5-8428-87fd8a72363c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:32.745373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 125/243] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:32 2022] GET /identity/v3/users/a341539db327425a8c3affa1e97f9c6f => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:32.749803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.749803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.749803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.769407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:32.770879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:32.817404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88d18c-85d4-4f29-be72-3c81411f99f3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:32.819946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:32.859216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fc736-12ba-4b8c-a64b-a4458562ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:32.860203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fc736-12ba-4b8c-a64b-a4458562ea7f None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.860357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 126/244] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:32.860817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fc736-12ba-4b8c-a64b-a4458562ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.861410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.861815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fc736-12ba-4b8c-a64b-a4458562ea7f None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.863166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fc736-12ba-4b8c-a64b-a4458562ea7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:32.866197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fc736-12ba-4b8c-a64b-a4458562ea7f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:32.921989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:32.947309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 119/245] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:32 2022] POST /identity/v3/users => generated 293 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:32.990815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a84a-dbc6-4ed8-be20-f32f71391d9a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.031221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.031759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.032037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.032306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.092495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a309d79-606f-4214-b109-248bbbca917e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.092817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a309d79-606f-4214-b109-248bbbca917e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.093084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a309d79-606f-4214-b109-248bbbca917e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.094584 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 120/246] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:33.099823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.100841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.101370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.127536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:33.128767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:33.143308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:33.144781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:33.219338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:33.235345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 121/247] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:33.274053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 127/248] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:33.278203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.280001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.383040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:33.397829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c7de1-6336-4193-a534-cb76fc4237d2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.410543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885da86e-792a-4b58-8e24-26d92274cbb4 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.441551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.442871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.443345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.444647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.460408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:33.460640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.461150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.461498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.461775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.462907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:33.464349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:33.467191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:33.468042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:33.468828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:33.469475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 128/249] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:33.471430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 122/250] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:33.477094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.479016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.507594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:33.618621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52b220-49bb-4748-840c-ea25b4d3f768 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.626453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b0302-7958-43f1-bd5b-fe3cbec3442f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.657960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.658974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.659459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.659924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.660848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] RBAC: Adding query filter params (role.id=176951ed62024e8493f28661d97c0cdd, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=a341539db327425a8c3affa1e97f9c6f) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:33.661405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:33.663160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:33.669559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 123/251] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/role_assignments?user.id=a341539db327425a8c3affa1e97f9c6f&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=176951ed62024e8493f28661d97c0cdd => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:33.670843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.672625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.673285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.673568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.673844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.674538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:33.677116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:33.677116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:33.689561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 129/252] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:33.693201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:33.695754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.874515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:33.891802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812e138-49f8-4f3d-b27f-dc9ce8fd8e40 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.915154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c097de-6a0e-423c-b57f-c47b00fcb16a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:33.941538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5662a-1209-4926-bc11-040c060b2cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.946262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5662a-1209-4926-bc11-040c060b2cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.946262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5662a-1209-4926-bc11-040c060b2cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.946262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5662a-1209-4926-bc11-040c060b2cd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.957453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5662a-1209-4926-bc11-040c060b2cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4db5662a-1209-4926-bc11-040c060b2cd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:33.963352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:33.964268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:33.965371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 124/253] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:33.970782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82710b-64a2-40b3-ad74-9b8e4fb9438d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.971072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82710b-64a2-40b3-ad74-9b8e4fb9438d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.971349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82710b-64a2-40b3-ad74-9b8e4fb9438d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.972911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 125/254] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:33.977699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82710b-64a2-40b3-ad74-9b8e4fb9438d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:33.990471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98405a-2463-485c-8fd8-419da601d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:33.992050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98405a-2463-485c-8fd8-419da601d424 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:33.992550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98405a-2463-485c-8fd8-419da601d424 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:33.993079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98405a-2463-485c-8fd8-419da601d424 None admin] PATH_INFO: `/v3/users/8db0453327314209918d011876fbd7d4` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:33.997117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98405a-2463-485c-8fd8-419da601d424 None admin] RBAC: Authorizing `identity:get_user(user_id=8db0453327314209918d011876fbd7d4)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:33.998840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98405a-2463-485c-8fd8-419da601d424 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:34.000694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 130/255] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/users/8db0453327314209918d011876fbd7d4 => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:34.003039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.003323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.003599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.039078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:34.040285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:34.137210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:34.189844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 131/256] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:34.190668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:34.191664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82710b-64a2-40b3-ad74-9b8e4fb9438d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:34.268781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:34.269739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.270152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.270554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.271559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:34.272246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:34.274319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:34.292991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 126/257] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:34.300167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:34.393287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:34.427566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ee112-e8f4-48d2-a92a-ece81219ae2d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:34.496621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280f4ba-6da5-436f-967a-78c05ded09c7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:34.498415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:34.499033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.499305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.499574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.566204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:34.566848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.567131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.567407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] PATH_INFO: `/v3/users/04866f6f226a40bc9a7a67f5f5c9dbaa` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.568563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] RBAC: Authorizing `identity:get_user(user_id=04866f6f226a40bc9a7a67f5f5c9dbaa)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:34.570361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:34.577599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 127/258] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:34 2022] GET /identity/v3/users/04866f6f226a40bc9a7a67f5f5c9dbaa => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:34.585699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:34.589109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:34.631040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:34.632792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:34.777205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd6d4-041a-49cc-ae1c-4aaab6dbdcb7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:34.786991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:34.813977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:34.814622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.814906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.815183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.816327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:34.817822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:34.820410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 128/259] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:34 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:34.821317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:34.831886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 132/260] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:34.832712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:34.935264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:34.972572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f267e90-e3fe-49fc-8248-b0755c4cdebe None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.010345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.011082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.011362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.011633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.022931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:35.033059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:35.033931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 129/261] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:35.034741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:35.057768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a41bbb-6ab4-4fda-974a-74654d767abe None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.112552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.113587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.114070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.114543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/04866f6f226a40bc9a7a67f5f5c9dbaa/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.117780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac8746c7d5e4606add57d76fb27178b, user_id=04866f6f226a40bc9a7a67f5f5c9dbaa, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.119469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:35.141564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1c069-f3b6-4242-b1e9-c3f5bb687f30 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.173734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:35.179103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.179792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.180068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.180334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.181515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.183144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:35.185673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 130/262] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:35 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:35.188943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:35.208141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 133/263] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:34 2022] PUT /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/04866f6f226a40bc9a7a67f5f5c9dbaa/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:35.209883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:35.328997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92de22-08d1-43c2-8e3f-c1b9d7eed537 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.330958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:35.368889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.368889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.368889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.368889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.369469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:35.369621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.371514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:35.388333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 131/264] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:35.389580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:35.460336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedc685-4d00-4173-becc-c4b93bafa840 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.500505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.501341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.501718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.502097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.502968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=8db0453327314209918d011876fbd7d4) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:35.503372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.505019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:35.513097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 134/265] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:35 2022] GET /identity/v3/role_assignments?user.id=8db0453327314209918d011876fbd7d4&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:35.518357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:35.594232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fc6e9-5bc2-46d0-9957-c4df521570ed None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.648547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.648547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.648547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.648547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] PATH_INFO: `/v3/users/8db0453327314209918d011876fbd7d4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.648964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] RBAC: Authorizing `identity:get_user(user_id=8db0453327314209918d011876fbd7d4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.648964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:35.649435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 132/266] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:35 2022] GET /identity/v3/users/8db0453327314209918d011876fbd7d4 => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:35.655477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:35.823204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0df77d-2d33-4265-93c9-23d0d6c17d12 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.829253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:35.869816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.870515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.870847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.871129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.956433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.957955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:36.093334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 135/267] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:36.099075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:36.242750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e621b-f8da-41a1-87fd-9ef3cbf9003d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:36.276007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:36.292332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4760b09-70ac-4f0e-b02c-97257e49adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:36.303240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4760b09-70ac-4f0e-b02c-97257e49adcd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.303240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4760b09-70ac-4f0e-b02c-97257e49adcd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.303240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4760b09-70ac-4f0e-b02c-97257e49adcd None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.303240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4760b09-70ac-4f0e-b02c-97257e49adcd None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:36.303240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4760b09-70ac-4f0e-b02c-97257e49adcd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:36.303240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 133/268] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:36 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:36.305058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:36.315185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:36.340982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9d8d-224f-46ec-9152-e91239a349dd None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.341572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9d8d-224f-46ec-9152-e91239a349dd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.343216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef9d8d-224f-46ec-9152-e91239a349dd None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.343945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 134/269] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:36.349002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca560bd-c084-4b99-b724-b30221d25003 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.349467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca560bd-c084-4b99-b724-b30221d25003 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.350447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca560bd-c084-4b99-b724-b30221d25003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.370354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3ca560bd-c084-4b99-b724-b30221d25003 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:36.371786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3ca560bd-c084-4b99-b724-b30221d25003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:36.493770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 135/270] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:36.503607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897e7e7-06bd-4532-a3bf-6da2c17db35e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:36.536438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4257795-4a12-4ef0-95fc-cd4c7aca511f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.537129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4257795-4a12-4ef0-95fc-cd4c7aca511f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.538144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4257795-4a12-4ef0-95fc-cd4c7aca511f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.540236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 136/271] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:14:36.546021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bdb56-32f2-4b68-b505-8825a4f91961 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.546596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bdb56-32f2-4b68-b505-8825a4f91961 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.547083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bdb56-32f2-4b68-b505-8825a4f91961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.559871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:36.560500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.560781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.561084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.561821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] RBAC: Adding query filter params (role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=8db0453327314209918d011876fbd7d4) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:36.562131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:36.563622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869771-1516-4ff5-ae3c-d423cc7118d3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:36.567058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b42bdb56-32f2-4b68-b505-8825a4f91961 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:36.568431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b42bdb56-32f2-4b68-b505-8825a4f91961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:36.578688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 136/272] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:36 2022] GET /identity/v3/role_assignments?user.id=8db0453327314209918d011876fbd7d4&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=97b084644adb4949ab768912c0bb75e8 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:36.678428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 137/273] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:36.710351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb47e1-6ea0-4fa9-9c91-b314d12b73c2 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.710632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb47e1-6ea0-4fa9-9c91-b314d12b73c2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.711067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb47e1-6ea0-4fa9-9c91-b314d12b73c2 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.712615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 137/274] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:36.724020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.724321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.724560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.746221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:36.747767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:36.815614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:36.823992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb47e1-6ea0-4fa9-9c91-b314d12b73c2 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:36.846162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 138/275] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:36.909830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:36.947153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb47e1-6ea0-4fa9-9c91-b314d12b73c2 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:36.984232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:36.985180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:36.985761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:36.986213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:36.998113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:37.004691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:37.005620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:37.007041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:37.012996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 138/276] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:37.015446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.046644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.133316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d873c-72b1-4b1d-bfed-b80e0728f84a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.172729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d236e6-f4d1-4c94-b703-3242252ad81e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.174403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.175034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.176070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.176070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.176263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:37.176510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.178025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.191723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 139/277] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:37.191723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.207039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:37.215019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.215718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.216883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.216883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.228060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:37.232891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:37.233847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:37.234862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 139/278] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:37.235559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.249868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:37.309671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82985e33-1d3f-4d90-9cd8-e4cbcc992cd9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.344303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae211-9853-4b5b-8793-c210211bd805 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.346559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.347190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.347465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.347740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.349500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.351567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.353230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 140/279] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:37.354049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.387051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.387681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.387975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.388223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] PATH_INFO: `/v3/users/e8773256cef142a5a1ef355620ff2cc6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.402410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] RBAC: Authorizing `identity:get_user(user_id=e8773256cef142a5a1ef355620ff2cc6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.403880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.405654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 140/280] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/users/e8773256cef142a5a1ef355620ff2cc6 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:37.410697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.455332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:37.471045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b78b9-4ba2-43b4-8f0d-579f04da420f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.488554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:37.507063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.507667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.507943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.508215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.508946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:37.509247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.520022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4ec5f-d4b7-4da4-8273-1e510e566fc3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.520153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.527361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 141/281] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:37.527361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.551162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.551802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.552086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.552390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.554162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.555639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.640213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f3e5-2e1e-4d8d-b3fc-d7706a375860 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.668668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 141/282] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:14:37 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:37.674794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.682013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.682644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.682923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.683194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.683963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, user.id=e8773256cef142a5a1ef355620ff2cc6, scope.system=all) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:37.684271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.686960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.700887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 142/283] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/role_assignments?user.id=e8773256cef142a5a1ef355620ff2cc6&scope.system=all&role.id=76c32cb614c145319ad750282d6d3551 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:37.827714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1cae0-0c5d-4837-9ab5-a67f1dcd5358 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:37.829418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:37.875897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581d846-c648-4aed-a35e-b4a258230538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:37.876980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d846-c648-4aed-a35e-b4a258230538 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.877457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d846-c648-4aed-a35e-b4a258230538 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.877951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581d846-c648-4aed-a35e-b4a258230538 None admin] PATH_INFO: `/v3/users/a341539db327425a8c3affa1e97f9c6f` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.879480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581d846-c648-4aed-a35e-b4a258230538 None admin] RBAC: Authorizing `identity:get_user(user_id=a341539db327425a8c3affa1e97f9c6f)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:37.881973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581d846-c648-4aed-a35e-b4a258230538 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:37.888654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 142/284] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/users/a341539db327425a8c3affa1e97f9c6f => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:37.890503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:37.943126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:38.015036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32923327-e8a1-4132-bfa8-b4013cff6280 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:38.025381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7ce0a-4cb4-4d52-8158-8b76623fb409 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.025667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7ce0a-4cb4-4d52-8158-8b76623fb409 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.025945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7ce0a-4cb4-4d52-8158-8b76623fb409 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.027608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 143/285] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:38.032545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.032827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.033126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.058228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:38.060921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:38.075376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:38.076024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.076301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.076574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.078431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:38.079815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:38.081487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 143/286] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:37 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:38.087702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:38.157628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 144/287] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:38.211493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3662a-ce70-4b1f-a9cf-4b910b88e68f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:38.223379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:38.249965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:38.250634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.250912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.251185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/a341539db327425a8c3affa1e97f9c6f/roles/176951ed62024e8493f28661d97c0cdd` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.253148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac8746c7d5e4606add57d76fb27178b, user_id=a341539db327425a8c3affa1e97f9c6f, role_id=176951ed62024e8493f28661d97c0cdd)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:38.254804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:38.367097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 144/288] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:14:38 2022] PUT /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b/users/a341539db327425a8c3affa1e97f9c6f/roles/176951ed62024e8493f28661d97c0cdd => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:14:38.551397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:38.806348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:38.806348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecdd18-1e08-4373-80bb-514c43c96f8c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:38.806348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:38.806348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:38.807241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:38.807832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:38.808791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:38.811432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:38.811432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:38.811432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:38.811432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 145/289] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:38.811955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:38.890661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f138385-0953-43d3-a2df-d6758276ed24 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:38.932403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:38.933052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.933330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.933602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.934269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:38.934566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:38.935931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:38.944749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 145/290] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:38.950630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:39.142719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e1de8-ad0f-4b2f-a2bc-a7dffc1d32d9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:39.188318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:39.188987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:39.189272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:39.189552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] PATH_INFO: `/v3/users/04866f6f226a40bc9a7a67f5f5c9dbaa` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:39.190717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] RBAC: Authorizing `identity:get_user(user_id=04866f6f226a40bc9a7a67f5f5c9dbaa)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:39.192240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:39.194088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 146/291] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:38 2022] GET /identity/v3/users/04866f6f226a40bc9a7a67f5f5c9dbaa => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:39.199756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:39.247959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:39.356602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ca7bb-3840-4be3-aadf-af98c9107ee6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:39.399028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:39.399973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:39.400465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:39.400978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:39.500165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:39.502026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:39.546717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:39.674446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 146/292] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:39.675820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:39.726318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:39.823331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e364cd-c91a-428c-8846-9dce06566b8b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:39.871096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f061d9-3edd-4e0b-b548-1da94c9a821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:39.871729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f061d9-3edd-4e0b-b548-1da94c9a821a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:39.872009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f061d9-3edd-4e0b-b548-1da94c9a821a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:39.872286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f061d9-3edd-4e0b-b548-1da94c9a821a None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:39.873435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f061d9-3edd-4e0b-b548-1da94c9a821a None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:39.874889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f061d9-3edd-4e0b-b548-1da94c9a821a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:39.876815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 147/293] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:39 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:39.885981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:40.076299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9bf06-2c6b-468f-b615-ad9242010b64 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:40.125449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:40.127542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:40.128404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.128912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.129390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.130382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=04866f6f226a40bc9a7a67f5f5c9dbaa) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:40.131379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:40.133105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675dd08-ce7e-44ba-afdb-0889751b9242 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:40.145469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 147/294] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:39 2022] GET /identity/v3/role_assignments?user.id=04866f6f226a40bc9a7a67f5f5c9dbaa&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=76c32cb614c145319ad750282d6d3551 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:40.364537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34065-b862-4805-aecd-60c427ea90c7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.364941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34065-b862-4805-aecd-60c427ea90c7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.365055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34065-b862-4805-aecd-60c427ea90c7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.367196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 148/295] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:40.376060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92176b-5455-496e-aa43-b5a924ead0b8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.377068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92176b-5455-496e-aa43-b5a924ead0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.377567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92176b-5455-496e-aa43-b5a924ead0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.416781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5b92176b-5455-496e-aa43-b5a924ead0b8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:40.418393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5b92176b-5455-496e-aa43-b5a924ead0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:40.505670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 148/296] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:40.586652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e098-c437-4748-9dc9-c9075eec183b None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.586971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e098-c437-4748-9dc9-c9075eec183b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.587205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e098-c437-4748-9dc9-c9075eec183b None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.588824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 149/297] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:40.594233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.594518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.594791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.622055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:40.623626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:40.672322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c5544-0725-4e14-81c3-1a49f2ff0da4 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.672322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c5544-0725-4e14-81c3-1a49f2ff0da4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.672322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c5544-0725-4e14-81c3-1a49f2ff0da4 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.674275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 150/298] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:40.679829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.680797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.681874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.702517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:40.703074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 149/299] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:40.704294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:40.780619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 151/300] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:40.839723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:40.898530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:40.947144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9de32-c6c6-4baf-aefb-80f830035242 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:40.986795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:40.987732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.988226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.988682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.991889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.001756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:41.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:41.008515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:41.009359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:41.011641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 150/301] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:41.017429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.022985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.102431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34405623-8cb4-45ba-88f2-79fe634bd2ea None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.122865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3edd47-6985-41b3-a621-a2d3da6bf251 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.136198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.136833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.137140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.137413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.139279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.153597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:41.153884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:41.155007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 152/302] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:41.155988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.161904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.163155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.163459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.163736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.164400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:41.164699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.166176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e9927-b9f3-4cdb-9786-714f93026988 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.175059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 151/303] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:41.175059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266704-b154-4c68-8ae7-0e000c50df5d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.175059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266704-b154-4c68-8ae7-0e000c50df5d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.175059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266704-b154-4c68-8ae7-0e000c50df5d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.175059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 152/304] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:41.175059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266704-b154-4c68-8ae7-0e000c50df5d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.244918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.259736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4cbec-7a37-46b8-b1a6-0cd4de78c211 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.280081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266704-b154-4c68-8ae7-0e000c50df5d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.293032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.293655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.293927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.294193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.295287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.296734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.298310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 153/305] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:41.299063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.316285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.317082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.317653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.317930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.318688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:41.319135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.320502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd4555-02a0-49a4-9a3d-121d9d890e36 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.329197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 153/306] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:41.330490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.330899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.331291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.348619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:41.350445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:41.384151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.398296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbfa76-b0f7-49e3-8b65-d81a6d9907af None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.416990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 154/307] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:41.417780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.452311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.452966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.453245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.453519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] PATH_INFO: `/v3/users/e8773256cef142a5a1ef355620ff2cc6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.454639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] RBAC: Authorizing `identity:get_user(user_id=e8773256cef142a5a1ef355620ff2cc6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.456073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.463061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 154/308] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/users/e8773256cef142a5a1ef355620ff2cc6 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:41.463061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.488257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.527626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc556cb2-fcaf-43bd-b656-0d06b8ca0bfd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.562245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.562847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.563119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.563364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06749224-a7ee-4ce1-ab94-29557edf1f18 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.563488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] PATH_INFO: `/v3/groups` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.563995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.565460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.597356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.597984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.598258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.598532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] PATH_INFO: `/v3/users/cf5ebfbb8c4842e78f32255058d204c6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.611934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] RBAC: Authorizing `identity:get_user(user_id=cf5ebfbb8c4842e78f32255058d204c6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.613424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.615460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 155/309] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/users/cf5ebfbb8c4842e78f32255058d204c6 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:41.616307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.676981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 155/310] 10.222.0.42 () {60 vars in 1134 bytes} [Tue Oct 4 10:14:41 2022] POST /identity/v3/groups => generated 229 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:14:41.676981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.710172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.716593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.737171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446122c5-6c54-4939-8ec4-2b75d2612842 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.775843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.776517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.776799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.777461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] PATH_INFO: `/v3/system/users/e8773256cef142a5a1ef355620ff2cc6/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.778587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e8773256cef142a5a1ef355620ff2cc6, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.779992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.790081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3b27d-398b-47ad-8c0e-01dab8689112 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.827110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca098e-680f-4287-817d-21421592258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.827727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca098e-680f-4287-817d-21421592258d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.828005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca098e-680f-4287-817d-21421592258d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.828276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca098e-680f-4287-817d-21421592258d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.829149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca098e-680f-4287-817d-21421592258d None admin] RBAC: Adding query filter params (role.id=176951ed62024e8493f28661d97c0cdd, user.id=cf5ebfbb8c4842e78f32255058d204c6, scope.system=all) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:41.829444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca098e-680f-4287-817d-21421592258d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.830906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca098e-680f-4287-817d-21421592258d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.837583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 156/311] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/role_assignments?user.id=cf5ebfbb8c4842e78f32255058d204c6&scope.system=all&role.id=176951ed62024e8493f28661d97c0cdd => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:41.838433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca098e-680f-4287-817d-21421592258d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.900846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 156/312] 10.222.0.42 () {58 vars in 1334 bytes} [Tue Oct 4 10:14:41 2022] PUT /identity/v3/system/users/e8773256cef142a5a1ef355620ff2cc6/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:41.965857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.981861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca098e-680f-4287-817d-21421592258d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:42.068091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:42.069002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:42.069285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:42.069553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:42.081497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:42.088168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:42.089350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:42.090373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 157/313] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:42.097109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:42.206727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de9b9b-f6a4-4372-9948-739eee38a5db None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:42.242992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:42.243522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:42.243797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:42.244063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:42.244731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:42.245064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:42.246444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:42.255410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 157/314] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:42.261740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:42.366738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a05c0-11ce-41d7-b50e-b7faf9d0fafc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:42.402101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:42.403562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:42.404055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:42.404536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] PATH_INFO: `/v3/users/a341539db327425a8c3affa1e97f9c6f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:42.405966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] RBAC: Authorizing `identity:get_user(user_id=a341539db327425a8c3affa1e97f9c6f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:42.407652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:42.409844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 158/315] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:42 2022] GET /identity/v3/users/a341539db327425a8c3affa1e97f9c6f => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:42.415813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:42.425418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:42.524005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b85da-0107-4210-af78-f0061b70b212 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:42.559427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:42.560035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:42.560311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:42.560580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:42.633453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:42.634914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:42.642976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:42.748009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 158/316] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:42.753479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:42.781704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:42.863627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1965df-202f-4369-abcf-2c046aaa6322 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:42.898313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6581a3-96d5-478a-99be-0e0cdbda3a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:42.898944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6581a3-96d5-478a-99be-0e0cdbda3a63 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:42.899219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6581a3-96d5-478a-99be-0e0cdbda3a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:42.899488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6581a3-96d5-478a-99be-0e0cdbda3a63 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:42.900606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6581a3-96d5-478a-99be-0e0cdbda3a63 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:42.902065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6581a3-96d5-478a-99be-0e0cdbda3a63 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:42.904041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 159/317] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:42 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:42.909978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:43.023626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbee3ad-a449-42c8-ac8e-51e586e65521 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:43.031819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:43.065649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:43.066258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:43.066535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:43.066804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:43.067623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] RBAC: Adding query filter params (role.id=176951ed62024e8493f28661d97c0cdd, scope.project.id=1ac8746c7d5e4606add57d76fb27178b, user.id=a341539db327425a8c3affa1e97f9c6f) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:43.067931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:43.069976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00447a-be1f-48f5-97f9-686eda5047d8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:43.076677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 159/318] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:14:42 2022] GET /identity/v3/role_assignments?user.id=a341539db327425a8c3affa1e97f9c6f&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=176951ed62024e8493f28661d97c0cdd => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:43.998467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684963e-74cc-4cf1-9b19-566fb322a338 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:43.999072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684963e-74cc-4cf1-9b19-566fb322a338 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:43.999072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684963e-74cc-4cf1-9b19-566fb322a338 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.000527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 160/319] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:44.006891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697da2f-d36b-4e51-ad65-e140e254e541 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.007406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697da2f-d36b-4e51-ad65-e140e254e541 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.007479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697da2f-d36b-4e51-ad65-e140e254e541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.026639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7697da2f-d36b-4e51-ad65-e140e254e541 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:44.027871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7697da2f-d36b-4e51-ad65-e140e254e541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:44.097224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 160/320] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:44.121425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539631c-3058-4b3f-a31c-e4a9d19e9530 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.121735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539631c-3058-4b3f-a31c-e4a9d19e9530 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.121976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539631c-3058-4b3f-a31c-e4a9d19e9530 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.123432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 161/321] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:44.128465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753038c-b6dd-44b3-b3cd-d0882b5342d4 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.128762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753038c-b6dd-44b3-b3cd-d0882b5342d4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.129083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753038c-b6dd-44b3-b3cd-d0882b5342d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.148330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2753038c-b6dd-44b3-b3cd-d0882b5342d4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:44.149544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2753038c-b6dd-44b3-b3cd-d0882b5342d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:44.229020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 161/322] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:44.298822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e0f5-3a23-4c30-b316-d525adb319ae None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.299136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e0f5-3a23-4c30-b316-d525adb319ae None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.299372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33e0f5-3a23-4c30-b316-d525adb319ae None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.300827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 162/323] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:44.306013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.306298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.306567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.324595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:44.325909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:44.349385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.364032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e0f5-3a23-4c30-b316-d525adb319ae None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.396212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 162/324] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:44.461411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.471894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33e0f5-3a23-4c30-b316-d525adb319ae None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:44.508170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:44.508808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.509127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.509400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.510506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:44.511947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:44.512143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.513665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 163/325] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:44.518741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.625029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c055602-65e4-48c4-9139-186bbbd862d7 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:44.626061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8f395-4d0b-4715-9d38-0ae229919ccc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:44.660827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:44.661487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.661766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.662528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.664259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:44.664881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.665161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.665430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] PATH_INFO: `/v3/groups` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.666059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:44.667498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:44.673652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:44.678619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:44.679595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 163/326] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:44.680766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.702798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.777218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62fccd-67ff-48f1-bdf3-73716afe4fed None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:44.777762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 164/327] 10.222.0.42 () {60 vars in 1134 bytes} [Tue Oct 4 10:14:44 2022] POST /identity/v3/groups => generated 226 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:14:44.778476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.800057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.832748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:44.833798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.834227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.834667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.847004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:44.852424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:44.853674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:44.854674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 164/328] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:44.861412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.906606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc2a0-37a0-4dec-b580-6a86e5001259 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:44.947060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:44.947682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.947960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.948230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.948928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:44.949265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:44.950670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:44.958301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.960439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 165/329] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:44.965819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.981642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5b1fa-1350-421a-8962-29bb20b0cfd5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:45.031385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:45.031385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:45.031385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:45.031385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:45.032186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:45.032186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:45.034069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:45.043145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 165/330] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:45.050490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:45.059194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:45.079143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ef362-16d5-4e7a-bb89-a942d0d41335 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:45.120940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:45.121609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:45.121883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:45.122152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] PATH_INFO: `/v3/users/cf5ebfbb8c4842e78f32255058d204c6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:45.124210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] RBAC: Authorizing `identity:get_user(user_id=cf5ebfbb8c4842e78f32255058d204c6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:45.124738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:45.127195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 166/331] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/users/cf5ebfbb8c4842e78f32255058d204c6 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:45.132492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:45.188145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19319ed5-2f59-419e-abcb-7f37a0f5b11b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:45.236578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:45.237252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:45.239278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:45.239278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] PATH_INFO: `/v3/users/e8773256cef142a5a1ef355620ff2cc6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:45.241879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] RBAC: Authorizing `identity:get_user(user_id=e8773256cef142a5a1ef355620ff2cc6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:45.242037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:45.243767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 166/332] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:45 2022] GET /identity/v3/users/e8773256cef142a5a1ef355620ff2cc6 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:45.249713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:45.273294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:45.274137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384eb077-7d10-44f1-9ff4-ad13dbcdd5ca None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:45.317236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e494-5988-43bc-9720-66133f05913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:45.318583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e494-5988-43bc-9720-66133f05913f None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:45.318583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e494-5988-43bc-9720-66133f05913f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:45.319151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e494-5988-43bc-9720-66133f05913f None admin] PATH_INFO: `/v3/system/users/cf5ebfbb8c4842e78f32255058d204c6/roles/176951ed62024e8493f28661d97c0cdd` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:45.320656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50e494-5988-43bc-9720-66133f05913f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cf5ebfbb8c4842e78f32255058d204c6, role_id=176951ed62024e8493f28661d97c0cdd)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:45.322339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50e494-5988-43bc-9720-66133f05913f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:45.386774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 167/333] 10.222.0.42 () {58 vars in 1334 bytes} [Tue Oct 4 10:14:45 2022] PUT /identity/v3/system/users/cf5ebfbb8c4842e78f32255058d204c6/roles/176951ed62024e8493f28661d97c0cdd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:14:45.395276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae229119-4971-416a-946d-f00ff7f1974c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:45.440094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:45.441199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:45.441706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:45.442185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:45.443792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] RBAC: Adding query filter params (role.id=76c32cb614c145319ad750282d6d3551, user.id=e8773256cef142a5a1ef355620ff2cc6, scope.system=all) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:45.444413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:45.446353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42983315-b8f4-44cf-ae4c-f675d130a012 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:45.463599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 167/334] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Oct 4 10:14:45 2022] GET /identity/v3/role_assignments?user.id=e8773256cef142a5a1ef355620ff2cc6&scope.system=all&role.id=76c32cb614c145319ad750282d6d3551 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:47.622252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a40f6-e480-4a22-a052-f4be944874ab None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:47.622704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a40f6-e480-4a22-a052-f4be944874ab None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:47.622788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a40f6-e480-4a22-a052-f4be944874ab None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:47.624194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 168/335] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:47.633601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bb1ca-5574-4336-b4d2-ed0797eb52c9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:47.633601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bb1ca-5574-4336-b4d2-ed0797eb52c9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:47.633601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bb1ca-5574-4336-b4d2-ed0797eb52c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:47.663787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f70bb1ca-5574-4336-b4d2-ed0797eb52c9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:47.666070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f70bb1ca-5574-4336-b4d2-ed0797eb52c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:47.755942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:47.765206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 168/336] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:47.804189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588273-fc2a-45d2-a523-4cbda9522549 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:47.804189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588273-fc2a-45d2-a523-4cbda9522549 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:47.804189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588273-fc2a-45d2-a523-4cbda9522549 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:47.804816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 169/337] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:47.809615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:47.809905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:47.810184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:47.837909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:47.839130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:47.918429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 169/338] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:48.242157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588273-fc2a-45d2-a523-4cbda9522549 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:48.260543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:48.348457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:48.493875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588273-fc2a-45d2-a523-4cbda9522549 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:48.516380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919247e1-6cdc-46d6-aa16-84e053a8976c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:48.561488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:48.573008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:48.574124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.574557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.574988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.581613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ddb75-386c-4760-aa37-7ea1c91fcf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:48.582420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ddb75-386c-4760-aa37-7ea1c91fcf06 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.582785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ddb75-386c-4760-aa37-7ea1c91fcf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.583158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ddb75-386c-4760-aa37-7ea1c91fcf06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.593364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:48.599052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:48.600551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:48.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:48.603942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 170/339] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:48.608523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ddb75-386c-4760-aa37-7ea1c91fcf06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:48.609602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-076ddb75-386c-4760-aa37-7ea1c91fcf06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:48.613896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:48.615925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:48.616756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 170/340] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:48.623517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ad062-1d40-4c56-841b-240e2ad0f626 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.623517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ad062-1d40-4c56-841b-240e2ad0f626 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.623517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ad062-1d40-4c56-841b-240e2ad0f626 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.624239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 171/341] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:48.629213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae971e1-7eb4-41f9-9d19-d8030eef5ad2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.629570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae971e1-7eb4-41f9-9d19-d8030eef5ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.630333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae971e1-7eb4-41f9-9d19-d8030eef5ad2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.632439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 172/342] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:48.634824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae971e1-7eb4-41f9-9d19-d8030eef5ad2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:48.717634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:48.734652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d859b7-92f5-48b9-8d80-19e9aa8090b1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:48.755561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae971e1-7eb4-41f9-9d19-d8030eef5ad2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:48.778089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:48.779023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.779504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.780080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.780970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:48.781492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:48.783258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c5ed0-378e-4a94-9e8f-8cf221889909 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:48.800085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 171/343] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:48.800085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.800085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.800085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.802011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:48.802686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.802962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.803233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.803940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:48.804239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:48.805677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390b17a-739f-46bc-a94b-b2e860be257d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:48.816376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 173/344] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:48.817783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:48.818091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:48.818362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:48.823081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:48.824672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:48.838468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:48.839767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:48.879931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:48.903042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 172/345] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:48.904309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:48.912325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 174/346] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:48.913120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.007113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.045983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc562a-79a9-4961-86ac-8ed8054bbdc6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.083971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b3b23-a4c3-4474-8af4-356338b684df None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.089202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.089864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.090140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.090405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.140044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.141061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.141550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.142034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.163964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:49.165492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:49.203407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.225911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:49.227860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:49.233111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.276606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 175/347] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:49.277393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.385487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114064fc-bbc8-48d3-a3c8-3abc066f5e6b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.421066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.421701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.421974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.422240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.433951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:49.438986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:49.439918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:49.440725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 176/348] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:49.441460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.452386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 173/349] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:49.456463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.484438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.577126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727e0e-9f3d-483e-8fd3-ff92b2ff1610 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.579944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.601801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35bea0-dcc9-4e38-b5ba-58802625cb89 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.622062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573c95-3d37-4959-9a7e-33668d8b51c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.622206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573c95-3d37-4959-9a7e-33668d8b51c9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.622531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573c95-3d37-4959-9a7e-33668d8b51c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.622716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573c95-3d37-4959-9a7e-33668d8b51c9 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.623809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f573c95-3d37-4959-9a7e-33668d8b51c9 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:49.626244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f573c95-3d37-4959-9a7e-33668d8b51c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:49.626944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 177/350] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:49.628096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56212121-29d0-4a5b-9fc8-bcfc3c7698c3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.628345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56212121-29d0-4a5b-9fc8-bcfc3c7698c3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.630910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56212121-29d0-4a5b-9fc8-bcfc3c7698c3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.631053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 178/351] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:49.631744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56212121-29d0-4a5b-9fc8-bcfc3c7698c3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.651060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.651060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.651060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.651060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.664252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:49.671493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:49.672449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 174/352] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:49.675675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.762898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.777067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56212121-29d0-4a5b-9fc8-bcfc3c7698c3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.830726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9ecd2-bae9-4c28-9bb0-b3575a36eabc None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.831234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.831832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.832111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.832385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.833095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:49.833401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:49.835829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:49.847589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 179/353] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:49.848499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:49.875870 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44d12c-f57f-4740-b4d0-83e7bd822316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:49.876517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44d12c-f57f-4740-b4d0-83e7bd822316 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.876797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44d12c-f57f-4740-b4d0-83e7bd822316 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.877122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44d12c-f57f-4740-b4d0-83e7bd822316 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.878290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44d12c-f57f-4740-b4d0-83e7bd822316 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:49.879769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d44d12c-f57f-4740-b4d0-83e7bd822316 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:49.882465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 175/354] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:49.883704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.883985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.884262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.904034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:49.905262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:49.907368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:49.982150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9ea5a-d939-4020-a1a8-eed39fe6e95e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:49.998966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 176/355] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:50.000133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.023712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.024346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.024615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.024900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.031510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.033241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.035967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 180/356] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.037360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.074029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.120098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e442f5-1a67-48e6-8a63-d1cfda429129 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.168977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.169624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.169902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.170179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.170995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:50.171309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.173705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.176040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acc5dd-4059-4aa7-8fc6-08f165d5cd18 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.180303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.185410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 177/357] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.186123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.217150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.218154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.218638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.219162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] PATH_INFO: `/v3/users/cf5ebfbb8c4842e78f32255058d204c6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.220510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] RBAC: Authorizing `identity:get_user(user_id=cf5ebfbb8c4842e78f32255058d204c6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.222197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.224403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 181/358] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/users/cf5ebfbb8c4842e78f32255058d204c6 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.225572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.310737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc144d-cc3b-4042-b59f-c7a8be229ca0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.322391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.335010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dc433-8f8b-45cc-8e56-9a4a6929cf8b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.350466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.351212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.351493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.352086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.352979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.354539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.356016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 178/359] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.356942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.376305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.377273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.377787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.378127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.378948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:50.379329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.382002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.387713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 182/360] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=97b084644adb4949ab768912c0bb75e8 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.388606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.445297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.496071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df444f5-489c-4d8a-875e-8e51ba2fc849 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.527782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab32d2-f917-4f5f-8d2f-1b44c34db501 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.544387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e873b-0c22-4d88-bb6c-adcfc8226c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.545558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e873b-0c22-4d88-bb6c-adcfc8226c96 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.546406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e873b-0c22-4d88-bb6c-adcfc8226c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.547182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e873b-0c22-4d88-bb6c-adcfc8226c96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.571848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.572469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.572744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.573039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.573771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] RBAC: Adding query filter params (role.id=176951ed62024e8493f28661d97c0cdd, user.id=cf5ebfbb8c4842e78f32255058d204c6, scope.system=all) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:50.574382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.575508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.592640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 183/361] 10.222.0.42 () {56 vars in 1308 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/role_assignments?user.id=cf5ebfbb8c4842e78f32255058d204c6&scope.system=all&role.id=176951ed62024e8493f28661d97c0cdd => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.593540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.610018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.663666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e873b-0c22-4d88-bb6c-adcfc8226c96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.665754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e873b-0c22-4d88-bb6c-adcfc8226c96 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.730828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee963591-8cae-4c5b-bd8a-c79dc95fabea None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.750544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.773962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.774610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.774886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.775156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.789290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:50.793420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 179/362] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:50.799330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c3710-f033-4e85-b3e4-ecf9f0ff7fc3 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:50.799501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:50.800344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 184/363] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:50.801196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.803128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c3710-f033-4e85-b3e4-ecf9f0ff7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.803609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c3710-f033-4e85-b3e4-ecf9f0ff7fc3 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.805544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 180/364] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:50.806537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c3710-f033-4e85-b3e4-ecf9f0ff7fc3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.886801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.933274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2a51-c1a1-4b69-890c-a9e2e5d81ab6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.955340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c3710-f033-4e85-b3e4-ecf9f0ff7fc3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.972077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.972695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.973002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.973276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.974011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:50.974431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.976047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.982327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 185/365] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=76c32cb614c145319ad750282d6d3551 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:50.983283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.006438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a66e9-f07c-489c-b6ed-2317b709a5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.007565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a66e9-f07c-489c-b6ed-2317b709a5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.007842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a66e9-f07c-489c-b6ed-2317b709a5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.008124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a66e9-f07c-489c-b6ed-2317b709a5a7 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.009425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a66e9-f07c-489c-b6ed-2317b709a5a7 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:51.011580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a66e9-f07c-489c-b6ed-2317b709a5a7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:51.014163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 181/366] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:51.016352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.016629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.016920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.044245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:51.045849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:51.095863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:51.115991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cca084-1237-41ec-9ca4-38fe7bd11ccb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:51.150131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 182/367] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:51.151359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.157573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:51.165542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.166512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.167043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.167551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.169671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:51.170228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:51.178258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:51.198485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 186/368] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:51.202671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.340164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c179282-a64f-4e0c-865c-8cb820b985e0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:51.387798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acdb80-a243-4d83-8a03-fb637dd46be4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:51.403111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.404053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.404486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.404996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.406937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:51.411147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:51.415507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 183/369] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:51.421337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.466076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.467865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.468441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.469883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.489291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:51.551589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:51.561532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:51.563369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:51.590781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7249f-2405-41ea-b235-e9ed481f370b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:51.633627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.634487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.634832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.635191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.636732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=1ac8746c7d5e4606add57d76fb27178b) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:51.637215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:51.638735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:51.647309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 184/370] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=76c32cb614c145319ad750282d6d3551 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:51.648438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.731828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 187/371] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:51.746220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.848033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:51.854107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f6ddd-889c-4c00-ae92-8bdaccdd3172 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:51.899536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.900289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.900582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.900877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.909270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:51.917091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b805428-944f-495d-91aa-2d721913ba9b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:51.925548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:51.930559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:51.931459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:51.932742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 185/372] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:51.937358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:51.960566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:51.961263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:51.961554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:51.961821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:51.963177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:51.964562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:51.966369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 188/373] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:51.972601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:52.089004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506368be-0e95-46b8-aad8-40a240797bbf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:52.132311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:52.132982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.133261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.133538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:52.134951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:52.136394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:52.138283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 189/374] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:52.144071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:52.157407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d591279f-7e43-458f-a12e-1c0a2dd2213c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:52.176921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:52.234025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:52.234745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.235085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.235360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:52.236084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:52.236389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:52.237927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:52.253890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 186/375] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:52.254729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05e4e1-a332-42b0-9401-539169884bf5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:52.278774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:52.295204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:52.295973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.296316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.296715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:52.297604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=1ac8746c7d5e4606add57d76fb27178b) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:52.297972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:52.299492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:52.308531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 190/376] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:14:52 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=97b084644adb4949ab768912c0bb75e8 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:52.330017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:52.467184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81fc9-5894-4e94-b8c7-143cbcc24aa7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:52.514924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:52.515701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.516073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.516441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:52.595573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:52.597220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:52.633126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:52.726480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 187/377] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 10:14:52.732152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:52.863262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:52.873161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56de33-0482-447f-a93b-3b3e573fb3e0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:52.916464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:52.917133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.917412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.917682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] PATH_INFO: `/v3/projects/76254fdd7dd646c69803abc10a472bc5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:52.918814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=76254fdd7dd646c69803abc10a472bc5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:52.920355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:52.922430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 191/378] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:52 2022] GET /identity/v3/projects/76254fdd7dd646c69803abc10a472bc5 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:52.927914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:53.043183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2605d84-ff80-4d09-9224-9c47b05f2ede None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:53.065759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:53.084670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ff35d-b45b-4c68-84a2-c341e43ade65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:53.085428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff35d-b45b-4c68-84a2-c341e43ade65 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.087151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff35d-b45b-4c68-84a2-c341e43ade65 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.087151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ff35d-b45b-4c68-84a2-c341e43ade65 None admin] PATH_INFO: `/v3/groups/1cc8d3fec523482a8af9c5a8459d0954` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.100242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff35d-b45b-4c68-84a2-c341e43ade65 None admin] RBAC: Authorizing `identity:get_group(group_id=1cc8d3fec523482a8af9c5a8459d0954)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:53.101769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ff35d-b45b-4c68-84a2-c341e43ade65 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:53.103758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 188/379] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:52 2022] GET /identity/v3/groups/1cc8d3fec523482a8af9c5a8459d0954 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:53.109556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:53.259408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc9414-d0d3-41fd-8d9b-9343bfb8e2a9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:53.309670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:53.310640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.311130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.311628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.312568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] RBAC: Adding query filter params (group.id=1cc8d3fec523482a8af9c5a8459d0954, role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=76254fdd7dd646c69803abc10a472bc5) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:53.313121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:53.314780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b771437-fa20-4bcb-b71d-523ce952d832 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:53.321880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 192/380] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:14:53 2022] GET /identity/v3/role_assignments?group.id=1cc8d3fec523482a8af9c5a8459d0954&scope.project.id=76254fdd7dd646c69803abc10a472bc5&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:53.338694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159730a8-d71f-4429-ada9-e16ebf7f72ba None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.339203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159730a8-d71f-4429-ada9-e16ebf7f72ba None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.339276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159730a8-d71f-4429-ada9-e16ebf7f72ba None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.341651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 189/381] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:53.346335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bdf97-1b3d-4a95-8403-96d95280e16b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.346935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bdf97-1b3d-4a95-8403-96d95280e16b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.347421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bdf97-1b3d-4a95-8403-96d95280e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.373929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0f7bdf97-1b3d-4a95-8403-96d95280e16b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:53.375141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0f7bdf97-1b3d-4a95-8403-96d95280e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:53.403528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:53.452512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 193/382] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:53.862986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7fa31-43ed-4874-9e04-051626ed3405 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.863327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7fa31-43ed-4874-9e04-051626ed3405 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.863660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7fa31-43ed-4874-9e04-051626ed3405 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.866134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 190/383] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:53.871815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.872247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.872322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.894400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:53.895589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:53.983483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 194/384] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:54.029567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7fa31-43ed-4874-9e04-051626ed3405 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.164826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7fa31-43ed-4874-9e04-051626ed3405 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:54.208582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:54.209830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.210110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.210387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.225620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:54.230452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:54.231357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:54.232158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:54.233383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 191/385] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:54.237793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.342275 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:54.358101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006cf20-53fe-4948-a74f-80d1a9aa17b2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:54.384988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.428603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:54.429599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.430099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.430588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.431464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:54.431986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:54.433614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:54.449754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 195/386] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:54.461961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.486508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:54.595318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7103a-96e7-4626-9413-95317710e3f7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:54.615456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809357d-b4ea-4266-b218-7448e852c583 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:54.635553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:54.636201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.636488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.636760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.638426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:54.639880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:54.641969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 196/387] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:54.647794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.671629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:54.698557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7681df-202f-4d58-820d-8ab6a3b62a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:54.699119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7681df-202f-4d58-820d-8ab6a3b62a2b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.699394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7681df-202f-4d58-820d-8ab6a3b62a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.699662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7681df-202f-4d58-820d-8ab6a3b62a2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.712930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7681df-202f-4d58-820d-8ab6a3b62a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4b7681df-202f-4d58-820d-8ab6a3b62a2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:54.718394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:54.719368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:54.720163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:54.720795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 192/388] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:54.722338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d4bb5-8449-4ed0-bada-0027899a5985 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.722338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d4bb5-8449-4ed0-bada-0027899a5985 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.722515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d4bb5-8449-4ed0-bada-0027899a5985 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.723963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 193/389] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:54.728753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d4bb5-8449-4ed0-bada-0027899a5985 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.774688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d445de-9246-4e1a-b78f-ff6cbf092476 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:54.814243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c155c-f537-4f9e-adee-02fad18e02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:54.815199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c155c-f537-4f9e-adee-02fad18e02ef None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.815758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c155c-f537-4f9e-adee-02fad18e02ef None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.816041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c155c-f537-4f9e-adee-02fad18e02ef None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.817210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c155c-f537-4f9e-adee-02fad18e02ef None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:54.818944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c155c-f537-4f9e-adee-02fad18e02ef None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:54.821051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 197/390] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:54.823035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.823365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.823685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.840429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d4bb5-8449-4ed0-bada-0027899a5985 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:54.845124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:54.846382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:54.853494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:54.879913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:54.880768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:54.881266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:54.881855 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:54.882836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:54.883282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:54.885165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:54.895114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 194/391] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:54.896938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.927788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 198/392] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:54.933468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:54.946397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:55.045739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d3db-6258-4b9d-bb80-7223d92c059e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.053478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d5b07-77b5-46a3-8dee-87fed7d18bfa None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.087104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fe898-bb64-4a7c-a904-c7905d80b6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.087813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fe898-bb64-4a7c-a904-c7905d80b6fa None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.088070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fe898-bb64-4a7c-a904-c7905d80b6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.088405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fe898-bb64-4a7c-a904-c7905d80b6fa None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.089585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fe898-bb64-4a7c-a904-c7905d80b6fa None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:55.091020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fe898-bb64-4a7c-a904-c7905d80b6fa None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:55.092932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c17fda-2805-4525-910b-0393cf5d5680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.093205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 199/393] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:54 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:55.094027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c17fda-2805-4525-910b-0393cf5d5680 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.094454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573973f4-c506-4953-bba8-67a1f521ab72 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.094730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573973f4-c506-4953-bba8-67a1f521ab72 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.094937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c17fda-2805-4525-910b-0393cf5d5680 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.095032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573973f4-c506-4953-bba8-67a1f521ab72 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.095488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c17fda-2805-4525-910b-0393cf5d5680 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/groups/7005fc26ed334860af14a6a3a9d233e3/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.097924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c17fda-2805-4525-910b-0393cf5d5680 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0ee695ae09442182b5b3e6795cbcc8, group_id=7005fc26ed334860af14a6a3a9d233e3, role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:55.099728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c17fda-2805-4525-910b-0393cf5d5680 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:55.101020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 200/394] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:55.102100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573973f4-c506-4953-bba8-67a1f521ab72 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.173002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 195/395] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Oct 4 10:14:54 2022] PUT /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/groups/7005fc26ed334860af14a6a3a9d233e3/roles/97b084644adb4949ab768912c0bb75e8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:55.174745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.175264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.175738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.214267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:55.216968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:55.222099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:55.266029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573973f4-c506-4953-bba8-67a1f521ab72 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.267981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:55.311259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.312487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.313034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.313543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.315134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:55.316965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:55.319257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 201/396] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:55.320464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.337994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 196/397] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:55.339295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.438944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeb671-fcc1-4b00-bf9e-25757267f3f0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.469980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db8545-527d-4c4a-8418-4c51841ebfbe None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.478060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.478966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.479451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.479921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.494222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:55.499515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:55.500472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:55.501320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:14:55.508924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.508924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.509181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.509449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/groups/7005fc26ed334860af14a6a3a9d233e3/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.509915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 202/398] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:55.510953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.511347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a0ee695ae09442182b5b3e6795cbcc8, group_id=7005fc26ed334860af14a6a3a9d233e3, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:55.512957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:55.526216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:55.587599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 197/399] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Oct 4 10:14:55 2022] PUT /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8/groups/7005fc26ed334860af14a6a3a9d233e3/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:55.623869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2e738-5ccd-451a-a15d-74d029845391 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.664206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.664527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.665165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.665447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.665718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.666388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:55.666691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:55.668095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:55.678862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 203/400] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:55.687067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.709249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:55.816700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:55.848593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a50404-a9cd-4f76-8202-c027ffe88e36 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.866440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df84574-2d2e-4801-a8d9-91b2bf81c1b5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:55.891404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.892656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.893587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.894336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.895695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:55.897750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:55.901123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 204/401] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:55.906223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:55.908443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:55.909615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.910087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.910590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.925974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:55.932020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:55.932944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:55.933765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:55.940002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 198/402] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:55.941448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.082398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.102826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2a543-7552-4f52-b1c5-6c39d7fdae66 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.128185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbf2da-d896-4537-9dfd-2f2a570d6133 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.152551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e7310-2298-4c20-9aef-d418d5ffbd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.153338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e7310-2298-4c20-9aef-d418d5ffbd4f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.153735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e7310-2298-4c20-9aef-d418d5ffbd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.154026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e7310-2298-4c20-9aef-d418d5ffbd4f None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.155149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e7310-2298-4c20-9aef-d418d5ffbd4f None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.156824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e7310-2298-4c20-9aef-d418d5ffbd4f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.158920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 205/403] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:56.165496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.165651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c9794-8841-4885-9795-33ff72e18d83 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.165905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c9794-8841-4885-9795-33ff72e18d83 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.166100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.166205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c9794-8841-4885-9795-33ff72e18d83 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.166376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.166654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.167319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:56.167662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 206/404] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:56.167771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.168145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c9794-8841-4885-9795-33ff72e18d83 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.173168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e14569-e472-4ee6-a8f7-f7a17d4bbd76 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.181724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 199/405] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:56.182948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.183331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.183612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.189201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.203343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:56.204531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:56.285655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c9794-8841-4885-9795-33ff72e18d83 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.304371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 200/406] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:56.305630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.335464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.336138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.336418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.336695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b/groups/7005fc26ed334860af14a6a3a9d233e3/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.338727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac8746c7d5e4606add57d76fb27178b, group_id=7005fc26ed334860af14a6a3a9d233e3, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.340169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.397963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.432897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21dfea-c826-4f02-b702-a488cfcb086a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.453736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 207/407] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Oct 4 10:14:56 2022] PUT /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b/groups/7005fc26ed334860af14a6a3a9d233e3/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:56.483498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.484253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.484424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.484982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.487018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.488540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.490682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 201/408] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:56 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:56.496736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.593369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.636732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57628fac-0d60-408f-9a1b-3d506242ea1f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.686264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.689347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.690057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.690344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.690620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.691870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.694185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.697526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 208/409] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:56 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:56.703622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.830122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd82bcd-b308-4f2d-ac89-65e58807bd4a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.843065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35eaa64-0928-42f9-ab95-36cee3583950 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.874250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.875136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.875414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.875686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.878778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.891104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:56.895396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:56.898528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:56.899538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:14:56.900344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:56.900979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.900979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.900979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b/groups/7005fc26ed334860af14a6a3a9d233e3/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.901164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 202/410] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:56.903273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ac8746c7d5e4606add57d76fb27178b, group_id=7005fc26ed334860af14a6a3a9d233e3, role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.903479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.905124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.909281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.991168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 209/411] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Oct 4 10:14:56 2022] PUT /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b/groups/7005fc26ed334860af14a6a3a9d233e3/roles/97b084644adb4949ab768912c0bb75e8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:57.040653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1ce80-5d0f-4a24-87fc-4084e336b550 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:57.088455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:57.089402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:57.089830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:57.090252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:57.091112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:57.091637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:57.093260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:57.104193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 203/412] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:57.113375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:57.238213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163d2b-7397-44d1-9b5a-97e56ed583c5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:57.275445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:57.276384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:57.276807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:57.277247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] PATH_INFO: `/v3/groups/1cc8d3fec523482a8af9c5a8459d0954` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:57.278535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] RBAC: Authorizing `identity:get_group(group_id=1cc8d3fec523482a8af9c5a8459d0954)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:57.280133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:57.281871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 210/413] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:57 2022] GET /identity/v3/groups/1cc8d3fec523482a8af9c5a8459d0954 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:57.288990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:57.351221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:57.403604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b2d5-38c4-4f8f-964e-0491c33a95a3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:57.455445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac67ef2-2b3c-4edc-99a6-1ce3189a3390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:57.456330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac67ef2-2b3c-4edc-99a6-1ce3189a3390 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:57.456750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac67ef2-2b3c-4edc-99a6-1ce3189a3390 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:57.457178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac67ef2-2b3c-4edc-99a6-1ce3189a3390 None admin] PATH_INFO: `/v3/projects/76254fdd7dd646c69803abc10a472bc5` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:57.458552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac67ef2-2b3c-4edc-99a6-1ce3189a3390 None admin] RBAC: Authorizing `identity:get_project(project_id=76254fdd7dd646c69803abc10a472bc5)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:57.460305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac67ef2-2b3c-4edc-99a6-1ce3189a3390 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:57.462115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 204/414] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:57 2022] GET /identity/v3/projects/76254fdd7dd646c69803abc10a472bc5 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:14:57.468088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:57.565989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:57.619653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328e1e6-ca68-42ca-b9d8-5522b4dd87ed None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:57.674768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa135f7-c215-4915-ba0f-dc184bc619b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:57.675553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa135f7-c215-4915-ba0f-dc184bc619b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:57.675839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa135f7-c215-4915-ba0f-dc184bc619b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:57.676118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa135f7-c215-4915-ba0f-dc184bc619b9 None admin] PATH_INFO: `/v3/projects/76254fdd7dd646c69803abc10a472bc5/groups/1cc8d3fec523482a8af9c5a8459d0954/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:57.678752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa135f7-c215-4915-ba0f-dc184bc619b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=76254fdd7dd646c69803abc10a472bc5, group_id=1cc8d3fec523482a8af9c5a8459d0954, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:57.680359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa135f7-c215-4915-ba0f-dc184bc619b9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:57.795843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 211/415] 10.222.0.42 () {58 vars in 1442 bytes} [Tue Oct 4 10:14:57 2022] PUT /identity/v3/projects/76254fdd7dd646c69803abc10a472bc5/groups/1cc8d3fec523482a8af9c5a8459d0954/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:14:58.044809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b33e00-d36a-4fd4-add3-19991a2f8a7a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.045389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b33e00-d36a-4fd4-add3-19991a2f8a7a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.045389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b33e00-d36a-4fd4-add3-19991a2f8a7a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.047112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 205/416] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:58.052765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf51e-445b-4a7c-b9e2-96e3b29d9d81 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.053162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf51e-445b-4a7c-b9e2-96e3b29d9d81 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.053402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf51e-445b-4a7c-b9e2-96e3b29d9d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.081929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-13baf51e-445b-4a7c-b9e2-96e3b29d9d81 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:58.083577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-13baf51e-445b-4a7c-b9e2-96e3b29d9d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:58.164549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 212/417] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:14:58.243943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72334cbd-114a-4c78-99c7-19a2b6b3893d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.244241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72334cbd-114a-4c78-99c7-19a2b6b3893d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.244525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72334cbd-114a-4c78-99c7-19a2b6b3893d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.246198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 206/418] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:58.251253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.251735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.252229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.272786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:58.274303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:58.336590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:58.358643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 213/419] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:58.448605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72334cbd-114a-4c78-99c7-19a2b6b3893d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:58.579904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72334cbd-114a-4c78-99c7-19a2b6b3893d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:58.607029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:58.636597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:58.637270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.637548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.637822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.651568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:58.656533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:58.657474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:14:58.658276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:58.663704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 207/420] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:14:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:14:58.665008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:58.715677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:58.885059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c477a4-0cf5-4953-ac2c-64d8a7f047ec None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:58.887184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e9172-b0c8-4916-97db-25081fda1ddb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:58.938549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:58.939424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.939710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.939983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.954254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:14:58.954592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:58.955410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.955899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.956362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.957266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:58.957780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:14:58.959319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:14:58.960271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:14:58.961120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:14:58.962408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:58.968910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 208/421] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:14:58.970533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:58.983769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 214/422] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:14:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:58.987697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.077285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.094549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.170196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237aaf1-c2aa-475d-a02f-8cce7155c5ce None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.232341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371db712-3336-466b-aa28-46b4a0ac93f2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.271402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47337ea-b57a-4198-b10b-bb00ebfb5f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.272371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47337ea-b57a-4198-b10b-bb00ebfb5f86 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.272797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47337ea-b57a-4198-b10b-bb00ebfb5f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.273239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47337ea-b57a-4198-b10b-bb00ebfb5f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.297591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.298214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.298497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.298774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.299469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:14:59.299751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.301175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.328958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 209/423] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:59.343626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.353249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47337ea-b57a-4198-b10b-bb00ebfb5f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.354714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47337ea-b57a-4198-b10b-bb00ebfb5f86 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.402614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.459883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 215/424] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:59.464452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208e9b-e021-4910-839c-e977fa4091f6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.464452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208e9b-e021-4910-839c-e977fa4091f6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.464452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208e9b-e021-4910-839c-e977fa4091f6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.464452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 216/425] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:59.467823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208e9b-e021-4910-839c-e977fa4091f6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.502946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839d6e3-0bc3-477f-9a00-90fce5cf89ec None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.516699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.558219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4212b-ce3a-4234-904f-a2f08f61c163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.559426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4212b-ce3a-4234-904f-a2f08f61c163 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.559853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4212b-ce3a-4234-904f-a2f08f61c163 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.560269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4212b-ce3a-4234-904f-a2f08f61c163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.597242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208e9b-e021-4910-839c-e977fa4091f6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.652794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4212b-ce3a-4234-904f-a2f08f61c163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.654272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4212b-ce3a-4234-904f-a2f08f61c163 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.658436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1c01f-728c-4f4c-8309-8d4b31bcfa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.659190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c01f-728c-4f4c-8309-8d4b31bcfa99 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.659467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c01f-728c-4f4c-8309-8d4b31bcfa99 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.659792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1c01f-728c-4f4c-8309-8d4b31bcfa99 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.660989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1c01f-728c-4f4c-8309-8d4b31bcfa99 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.663633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1c01f-728c-4f4c-8309-8d4b31bcfa99 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.665807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 217/426] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:59 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:59.669274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.669706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.670084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.691118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:59.692507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:59.703005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.765885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 210/427] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:14:59.765885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92fcec-ab8d-4b91-be43-0c4750008348 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.765885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92fcec-ab8d-4b91-be43-0c4750008348 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.765885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92fcec-ab8d-4b91-be43-0c4750008348 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.765885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 211/428] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:59.765885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92fcec-ab8d-4b91-be43-0c4750008348 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.776474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.777004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 218/429] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:14:59.777873 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.888152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92fcec-ab8d-4b91-be43-0c4750008348 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.906766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e606068-3e9b-4c93-8e6e-d802c2402f4c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.925020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca20118-166a-4d9c-9408-f9b8c466f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.926040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca20118-166a-4d9c-9408-f9b8c466f2cb None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.926521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca20118-166a-4d9c-9408-f9b8c466f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.927550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca20118-166a-4d9c-9408-f9b8c466f2cb None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.929509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca20118-166a-4d9c-9408-f9b8c466f2cb None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.931188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca20118-166a-4d9c-9408-f9b8c466f2cb None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.937246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 212/430] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:14:59 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:59.938728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.940572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.941090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.945880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.952074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.952074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.952074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] PATH_INFO: `/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.952074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] RBAC: Authorizing `identity:get_project(project_id=4a0ee695ae09442182b5b3e6795cbcc8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.952074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.956548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 219/431] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:14:59 2022] GET /identity/v3/projects/4a0ee695ae09442182b5b3e6795cbcc8 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:14:59.957459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.962779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:59.964206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:59.976570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.039946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 213/432] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:00.041333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.065205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.080441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf6324-ae5c-4fe3-983d-d488ad0fa595 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.121495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.122358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.122640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.122910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.124104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:00.124682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:00.126256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:00.135363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 220/433] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:14:59 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=97b084644adb4949ab768912c0bb75e8 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:00.136235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.242572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7a12a-6552-4fc0-937b-25573a63b403 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.310169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.311184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.311666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.312140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.314436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:00.316136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:00.318977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.321911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 214/434] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:00.327835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.351015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270efa2-80f2-457c-845d-ac2a01898476 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.403603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.403603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.403603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.403603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.419963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:00.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:00.426166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:15:00.426971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:00.427667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 221/435] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:00.428634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.441356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.469106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6597d-235a-4d43-b9e6-46d1ae6aae0c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.503547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.504154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.504438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.504716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.505753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:00.506067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:00.507511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:00.513341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 215/436] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=4a0ee695ae09442182b5b3e6795cbcc8&role.id=76c32cb614c145319ad750282d6d3551 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:00.513953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.550267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935f246-1ed3-4c60-bd32-ff43fefa5499 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.591172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b6b44-b4b6-4bab-aad4-4a43f514aa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.591737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6b44-b4b6-4bab-aad4-4a43f514aa32 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.592023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6b44-b4b6-4bab-aad4-4a43f514aa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.592299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b6b44-b4b6-4bab-aad4-4a43f514aa32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.595629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.604920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b6b44-b4b6-4bab-aad4-4a43f514aa32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-be5b6b44-b4b6-4bab-aad4-4a43f514aa32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:00.611851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:00.613021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 222/437] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:00.613832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d952a-7ae8-4621-8bdb-39d7b61056e8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.614567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d952a-7ae8-4621-8bdb-39d7b61056e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.614567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d952a-7ae8-4621-8bdb-39d7b61056e8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.615556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 223/438] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:00.617659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d952a-7ae8-4621-8bdb-39d7b61056e8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.645012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5635b09-a904-458c-9b67-ec8079568739 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.687110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.687843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.688208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.688545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.689252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] RBAC: Adding query filter params (name=member) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:00.689549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:00.690939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244095c1-f088-4c57-9ebb-55b49f42f04d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:00.696907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.701110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 216/439] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:00.706321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.706321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.706321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.723533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:00.727398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:00.734581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d952a-7ae8-4621-8bdb-39d7b61056e8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.776513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.777231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.777505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.777773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.778466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:00.778765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:00.780131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:00.790500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 224/440] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:00.791335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.805220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 217/441] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:00.806481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:00.868237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.922394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11384e-d0b1-4753-922c-587dbe8609fd None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.928572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91123f7b-bc19-4115-9ff6-450c892790fc None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:00.963860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.964483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.964766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.965090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:00.966444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:00.973204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:00.974887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:00.974989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:00.975893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.048001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.049581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.056516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.058138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.119865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.170273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 218/442] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:01.171133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.172528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 225/443] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:01.177365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.205844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.279300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbca982-9e3c-4b5f-9782-ba9f0ba711e2 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.287442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a802f-da2d-4def-a000-a38f087f7a32 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.311863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.312477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.312753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.313057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.322886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.323507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.323606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:01.323777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.324039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.325165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.326607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.328190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 226/444] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:01.328278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:01.330215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:01.331279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 219/445] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:01.332141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.334081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.359262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.422603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.431123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93ee1-5789-4b1f-941e-d6c361b7c329 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.432614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccacb3d-0b49-4d12-bcbc-7c2e45e1b084 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.465101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.465733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.466011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.466284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] PATH_INFO: `/v3/projects/1ac8746c7d5e4606add57d76fb27178b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.467430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] RBAC: Authorizing `identity:get_project(project_id=1ac8746c7d5e4606add57d76fb27178b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.467930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.468623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.468877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.469016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.469293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.470666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.470781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 227/446] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/projects/1ac8746c7d5e4606add57d76fb27178b => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:01.471255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.471840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.478253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 220/447] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:01.478253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.568792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.574152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73c96-d0ce-472e-952a-0a32f09933b1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.575482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211f3e2-8d45-4ba7-9f8e-f3629b111f4c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.607282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.607905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.608187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.608479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] PATH_INFO: `/v3/groups/7005fc26ed334860af14a6a3a9d233e3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.608700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.609332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.609611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.609709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] RBAC: Authorizing `identity:get_group(group_id=7005fc26ed334860af14a6a3a9d233e3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.609876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.610543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:01.610845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.611120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.614161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.614374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 221/448] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/groups/7005fc26ed334860af14a6a3a9d233e3 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:01.614374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.626421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 228/449] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:01.627121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.643287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.713655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf6eef-cc29-4e6e-8e4e-5974441115e1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.726035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e02fc1-dca0-433a-a332-0d151a53bbe5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.752949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.753551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.753827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.754107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.754824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=1ac8746c7d5e4606add57d76fb27178b) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:01.755800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.757380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.762243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.763140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.763242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 222/450] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=97b084644adb4949ab768912c0bb75e8 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:01.763315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.763393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.763683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:01.764094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] RBAC: Adding query filter params (group.id=7005fc26ed334860af14a6a3a9d233e3, role.id=76c32cb614c145319ad750282d6d3551, scope.project.id=1ac8746c7d5e4606add57d76fb27178b) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:01.764402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:01.765876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:01.778323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 229/451] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/role_assignments?group.id=7005fc26ed334860af14a6a3a9d233e3&scope.project.id=1ac8746c7d5e4606add57d76fb27178b&role.id=76c32cb614c145319ad750282d6d3551 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:01.886638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.898735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71263ba-9803-4f29-b4cc-92e9804dbe4d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:01.939265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:01.940242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.940785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.941245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:02.020746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:02.023100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:02.112900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:02.123797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 223/452] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:02.129846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:02.154829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:02.235796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14084591-2bf2-4742-a83d-689a1c1d75ac None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:02.271778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:02.272344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:02.272629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:02.272930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] PATH_INFO: `/v3/projects/76254fdd7dd646c69803abc10a472bc5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:02.274061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] RBAC: Authorizing `identity:get_project(project_id=76254fdd7dd646c69803abc10a472bc5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:02.275512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:02.277280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 230/453] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:15:02 2022] GET /identity/v3/projects/76254fdd7dd646c69803abc10a472bc5 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:02.283043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:02.385125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef1f47-1f23-4252-8eaa-615486ee5101 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:02.418045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd062084-dc3d-4fc7-ba18-629caea511cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:02.418657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062084-dc3d-4fc7-ba18-629caea511cf None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:02.418934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062084-dc3d-4fc7-ba18-629caea511cf None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:02.419204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062084-dc3d-4fc7-ba18-629caea511cf None admin] PATH_INFO: `/v3/groups/1cc8d3fec523482a8af9c5a8459d0954` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:02.420276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd062084-dc3d-4fc7-ba18-629caea511cf None admin] RBAC: Authorizing `identity:get_group(group_id=1cc8d3fec523482a8af9c5a8459d0954)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:02.421742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd062084-dc3d-4fc7-ba18-629caea511cf None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:02.423275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 224/454] 10.222.0.42 () {56 vars in 1180 bytes} [Tue Oct 4 10:15:02 2022] GET /identity/v3/groups/1cc8d3fec523482a8af9c5a8459d0954 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:02.430164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:02.515562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:02.534134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b3569-85a6-43b2-b32a-e0218bedab30 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:02.561396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:02.561988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:02.562264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:02.562535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:02.563249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] RBAC: Adding query filter params (group.id=1cc8d3fec523482a8af9c5a8459d0954, role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=76254fdd7dd646c69803abc10a472bc5) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:02.563557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:02.564998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fa7b7-e39a-44d0-87e2-47a40ec8c73f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:02.570778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 231/455] 10.222.0.42 () {56 vars in 1376 bytes} [Tue Oct 4 10:15:02 2022] GET /identity/v3/role_assignments?group.id=1cc8d3fec523482a8af9c5a8459d0954&scope.project.id=76254fdd7dd646c69803abc10a472bc5&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:05.272203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ec37f-67d3-4a86-8a65-a8c24ab060ad None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.272723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ec37f-67d3-4a86-8a65-a8c24ab060ad None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.272800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ec37f-67d3-4a86-8a65-a8c24ab060ad None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:05.274434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 225/456] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:05.279657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586dc80-8b5e-4820-94e9-4d0830fabc58 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.280087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586dc80-8b5e-4820-94e9-4d0830fabc58 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.280257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586dc80-8b5e-4820-94e9-4d0830fabc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:05.310173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1586dc80-8b5e-4820-94e9-4d0830fabc58 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:05.311454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1586dc80-8b5e-4820-94e9-4d0830fabc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:05.403822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 232/457] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:05.445577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281adba-fed3-4fd7-b7a8-8c505794787a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.446015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281adba-fed3-4fd7-b7a8-8c505794787a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.446502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281adba-fed3-4fd7-b7a8-8c505794787a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:05.449397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 226/458] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:05.454717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6dba-aaa4-4162-a648-6e8612be52be None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.455067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6dba-aaa4-4162-a648-6e8612be52be None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.455322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e6dba-aaa4-4162-a648-6e8612be52be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:05.485858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a28e6dba-aaa4-4162-a648-6e8612be52be None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:05.487127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a28e6dba-aaa4-4162-a648-6e8612be52be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:05.582725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 233/459] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:05.651126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e92f2-025f-45dc-acf7-965072480593 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.651735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e92f2-025f-45dc-acf7-965072480593 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.652235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e92f2-025f-45dc-acf7-965072480593 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:05.653814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 227/460] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:05.658965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.659258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.659536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:05.689070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:05.690254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:05.698898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:05.779136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e92f2-025f-45dc-acf7-965072480593 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:05.798016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 234/461] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:05.882169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:05.906333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:05.950408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e92f2-025f-45dc-acf7-965072480593 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:05.989927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:05.991154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:05.991435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:05.991708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.002716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.004183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-e6284b61-acca-47fb-ae1e-86e10229d4b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:06.020010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:06.021107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:06.021960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:06.021960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:06.021960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:06.021960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 228/462] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:06.027573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ff0b7-37c6-4a09-a7ef-d822e5caa648 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.027573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ff0b7-37c6-4a09-a7ef-d822e5caa648 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.027573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ff0b7-37c6-4a09-a7ef-d822e5caa648 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.027573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 229/463] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:06.028959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ff0b7-37c6-4a09-a7ef-d822e5caa648 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.041922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9c32b-8600-4482-9f54-dc1c68b6831d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.082006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.082641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.082913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.084289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.098466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.100478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.103003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-423ef5bc-d59b-4ab1-8829-98fee5ec3637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:06.120258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:06.122167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:06.123448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:06.123448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:06.123448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:06.128882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 235/464] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:06.128882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.128882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.128882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.146757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:06.147999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:06.161288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516ff0b7-37c6-4a09-a7ef-d822e5caa648 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.172720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:06.197931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.198530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.198920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.199113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.199698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:06.200003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.201413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.217229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 230/465] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:06.217229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.225600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 236/466] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:06.226444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.338941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40584542-9aa3-4d61-a6a4-d9bc02c9a24d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.345958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708280c-92a0-4b95-ae80-be1a668f9aef None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.366387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:06.377522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.378608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.382789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.382789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.390856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.391148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.391757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.392128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.392497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.393275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:06.393472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.393634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.395656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:06.415831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:06.416790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:06.417866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:06.417866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:06.417866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:06.417866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 237/467] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:06.417866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.418214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 231/468] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:06.419247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.450619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:06.538748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565581-8757-4be8-b9df-2368dd544946 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.544380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c6321-27e7-4e8a-bf9d-b46f3a6b2293 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.574716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.575954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.576389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.577066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.577972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.579775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.580626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.581266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.581549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.581825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.582469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.584420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.698773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 238/469] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:15:06 2022] POST /identity/v3/users => generated 246 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:06.699535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.734874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 232/470] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:15:06 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:06.736301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:06.760425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:06.924160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d4213-1e8d-499a-8bf9-023e07c448e2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.947130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:06.966799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29b0a0-4029-46c5-9f25-8bdd0d7eb677 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:06.968095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:06.968721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.969027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.969307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.970030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:06.970343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:06.971753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8e927-ef08-41e5-9936-bb78ab10eaf1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:06.996899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 239/471] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:06.996899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44265-b6c5-48d7-a0c8-2e670516832f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:06.996899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44265-b6c5-48d7-a0c8-2e670516832f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:06.996899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d44265-b6c5-48d7-a0c8-2e670516832f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:06.996899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 240/472] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:06.998480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44265-b6c5-48d7-a0c8-2e670516832f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:07.016043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:07.021186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.021186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.021186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.028945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:07.030440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-b1f7fdff-d924-49ca-90bb-f2ecc9b748aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:07.048291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:07.049275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:07.050114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:07.050114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:07.050114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:07.050114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 233/473] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:07.052424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.052424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.052424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.074764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:07.075960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:07.124021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:07.137253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d44265-b6c5-48d7-a0c8-2e670516832f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:07.153975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 234/474] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:07.154893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:07.179654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:07.180618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.181425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.181894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.182866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:07.184509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:07.256529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:07.292909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1121c7-9e0d-43c3-b128-88b492e5d455 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:07.313678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 241/475] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:15:06 2022] POST /identity/v3/users => generated 248 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:15:07.339813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:07.340879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.341394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.341885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.342771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:07.343265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:07.344845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:07.363423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 235/476] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:07.369958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:07.472675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:07.502067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bef4f-f79f-472a-a090-675a6dcaf223 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:07.552929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:07.554258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.554767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.555248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.556115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:07.558783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:07.632792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c30e7-096d-4079-964e-9749cab7abf1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:07.642477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 242/477] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:15:07 2022] POST /identity/v3/users => generated 249 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:07.673061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:07.683524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:07.684656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.685236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.685740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.686685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:07.688330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:07.727758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.sql.core [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 10:15:07.727758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 10:15:07.727758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [parameters: {'id': '6844f49ef06f4622b07628dfa2ea8946', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 10:15:07.727758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81560) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application result.read() Oct 04 10:15:07.759903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:15:07.760839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application yield resource Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise self.value Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application session.commit() Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 10:15:07.761804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self.session.flush() Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application compat.raise_( Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 10:15:07.762684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:15:07.763565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application result.read() Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:15:07.764385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [parameters: {'id': '6844f49ef06f4622b07628dfa2ea8946', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:07.765220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 10:15:07.766021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:07.768843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 236/478] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Oct 4 10:15:07 2022] POST /identity/v3/roles => generated 144 bytes in 296 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:07.774852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:07.785358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:07.915943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1238b5-72f9-45ff-9310-612046ac80e9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:07.958799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5703ec-11ca-4222-bfe7-051433e05d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:07.959444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5703ec-11ca-4222-bfe7-051433e05d75 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:07.959722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5703ec-11ca-4222-bfe7-051433e05d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:07.960030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5703ec-11ca-4222-bfe7-051433e05d75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:07.972628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5703ec-11ca-4222-bfe7-051433e05d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3a5703ec-11ca-4222-bfe7-051433e05d75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:07.978581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:07.979501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:07.980814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 243/479] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:07.987615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:08.099576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f11f67-318f-46f8-82a4-5c4b13a53cc9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:08.136962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:08.137576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:08.137848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:08.138176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:08.138843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:08.139223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:08.140711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3f863-4dea-43a3-b22b-e7839299c1d8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:08.149791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 237/480] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:09.299400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eef4e4-25ca-4335-959a-1e4b94cb734d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:09.300108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eef4e4-25ca-4335-959a-1e4b94cb734d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:09.300108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eef4e4-25ca-4335-959a-1e4b94cb734d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:09.301546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 244/481] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:09.314254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49adae8-12b8-49d9-8628-69b0ec09f4a6 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:09.314788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49adae8-12b8-49d9-8628-69b0ec09f4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:09.314788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49adae8-12b8-49d9-8628-69b0ec09f4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:09.342233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e49adae8-12b8-49d9-8628-69b0ec09f4a6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:09.343474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e49adae8-12b8-49d9-8628-69b0ec09f4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:09.408794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:09.435162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 238/482] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:09.542258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a832d3-366b-45f3-a9b7-33078f43e0b9 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:09.542258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a832d3-366b-45f3-a9b7-33078f43e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:09.542258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a832d3-366b-45f3-a9b7-33078f43e0b9 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:09.543139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 245/483] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:09.547514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:09.547799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:09.548078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:09.578483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:09.579604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:09.683099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 239/484] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:15:09.771901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a832d3-366b-45f3-a9b7-33078f43e0b9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:09.856936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:09.911042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a832d3-366b-45f3-a9b7-33078f43e0b9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:09.955960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:09.956582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:09.956872 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:09.957156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:09.969228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:09.975114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:09.976092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:09.977515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 246/485] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:09.981962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.038784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.127365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeefb46-5cd3-4f64-b89b-69f0081499b1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.148702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:10.152287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f7d0-0bf2-4d82-8ac7-1ab27396bba3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.175212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.176315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.176815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.177573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.178859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:10.179521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:10.180083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.182090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fe7ca-283c-4de3-9dbe-2b6b067241e4 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:10.190593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519509a-c91e-4759-9607-fcbf27f12a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.191429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519509a-c91e-4759-9607-fcbf27f12a5c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.191811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519509a-c91e-4759-9607-fcbf27f12a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.192190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519509a-c91e-4759-9607-fcbf27f12a5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.194079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 240/486] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:10.195473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752de4a7-f8a6-49b9-ace2-f51506b1327d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.195891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752de4a7-f8a6-49b9-ace2-f51506b1327d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.196327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752de4a7-f8a6-49b9-ace2-f51506b1327d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.198280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 241/487] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:10.200078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752de4a7-f8a6-49b9-ace2-f51506b1327d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.211011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519509a-c91e-4759-9607-fcbf27f12a5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4519509a-c91e-4759-9607-fcbf27f12a5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:10.216133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:10.217249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:10.218266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:10.220260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 247/488] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:10.221664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457adf2-7163-488b-a373-709ef8008170 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.222048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457adf2-7163-488b-a373-709ef8008170 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.222427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457adf2-7163-488b-a373-709ef8008170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.247089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2457adf2-7163-488b-a373-709ef8008170 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:10.247777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2457adf2-7163-488b-a373-709ef8008170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:10.336248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 248/489] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:10.337289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457adf2-7163-488b-a373-709ef8008170 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.340216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752de4a7-f8a6-49b9-ace2-f51506b1327d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.387354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.388652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.389101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.389920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.403640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.405968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:10.425386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:10.426321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:10.427732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:10.427732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:10.427732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:10.427732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 242/490] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:10.433159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.462144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457adf2-7163-488b-a373-709ef8008170 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.462651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:10.479201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:10.504534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.509151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.509434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.509709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.511891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:10.511891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.514801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:10.533076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 249/491] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:10.538932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.579590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63d69-6851-4d8a-adbf-43c2f7c2c3b8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.639390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.639932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.640213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.640490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.641167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:10.641475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.642868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:10.655711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 243/492] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:10.661154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.674760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e493ac-8d1e-410b-99b9-7dd8cbd603e7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.731186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.732066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.732432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.733110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.748593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.750093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:10.767410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-14826bbd-b73b-4411-9d77-f7976a36e14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:10.770671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:10.771614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:10.772444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:10.772444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:10.772444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:10.773383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 250/493] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:10.774703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9fa0c-c5a3-4b9e-8e44-466fcde695e8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.774983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9fa0c-c5a3-4b9e-8e44-466fcde695e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.775670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9fa0c-c5a3-4b9e-8e44-466fcde695e8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.777416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 251/494] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:10.778300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9fa0c-c5a3-4b9e-8e44-466fcde695e8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.783511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549b364-46c5-47be-8ff3-8576353a63af None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.809361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:10.817013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.818715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.818997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.819270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.827043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.828505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:10.842259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:10.843372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:10.844288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 244/495] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:10.844875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:10.902500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9fa0c-c5a3-4b9e-8e44-466fcde695e8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.939744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791891cf-8f0c-41a9-a2f6-2c328147a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.940381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791891cf-8f0c-41a9-a2f6-2c328147a460 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.940665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791891cf-8f0c-41a9-a2f6-2c328147a460 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.940976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791891cf-8f0c-41a9-a2f6-2c328147a460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.952697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791891cf-8f0c-41a9-a2f6-2c328147a460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:10.953237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6b9e7-d0d5-4e5f-aff7-bc89945d1eb9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-791891cf-8f0c-41a9-a2f6-2c328147a460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:10.957474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:10.958748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 252/496] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:10.961758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.962476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.962476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.980760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:10.982078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:10.993121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:10.994014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:10.994014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:10.994300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:10.994873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:10.995251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:10.996535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.012835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 245/497] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:11.013726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.024334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.068969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 253/498] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:11.068969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.092976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.132098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a7348-d627-4996-bb71-c484286a2285 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.173991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.174523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.174804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.175127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.175864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:11.176167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.177721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.189295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19f5c0-19c2-4711-a5c9-d6b4b3d89acd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.216054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 246/499] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:11.216924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.231660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.232276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.232764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.233077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.234134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:11.234447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.236135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32001055-5c72-4739-b9a1-3d21631a2752 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.249723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 254/500] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:11.249723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7f51-629d-472c-9bc0-19a1560d253e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.249723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7f51-629d-472c-9bc0-19a1560d253e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.249723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7f51-629d-472c-9bc0-19a1560d253e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.250308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 255/501] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:11.251027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7f51-629d-472c-9bc0-19a1560d253e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.294725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.330666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd5f96-40f5-4f1b-a826-a78826e5d14c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.342013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.363479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7f51-629d-472c-9bc0-19a1560d253e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.365252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.365886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.366164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.366440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.375218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.376688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:11.388355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:11.389480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 247/502] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:11.390416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.401538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.402143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.402416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.402679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.471709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.473120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.488119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87996ae-690a-4448-9f55-d225a2fde4b5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.512963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.521646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.522270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.522550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.522824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.532887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.534340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-28fa216c-04b5-482f-aec3-fc8ea7773642 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:11.548351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:11.549538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:11.550712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:11.550712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:11.550712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:11.550712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 248/503] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:11.550996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.551076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.551390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.554223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.570208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:11.571411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:11.574419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 256/504] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:11.575250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.638051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 249/505] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:11.638901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.679182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e1d46-692b-49bb-88ff-3407a4d92bd5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.714976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.715855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.716301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.716711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.728392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:11.733323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:11.734227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:11.735030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 257/506] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:11.735680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.759058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c778f8-22c7-4fa1-a8ab-e0a0395cdc5d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.768810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.783109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:11.793971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.794584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.794861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.795144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.795859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:11.796421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.798564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.820403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 250/507] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:11.820403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.850271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819380d-c0f3-4b72-b461-4a808647d103 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:11.885363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:11.885966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:11.886242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:11.886512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:11.887143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:11.887441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:11.888798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:11.900591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 258/508] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:11.900591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:11.932733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdce91-8e13-419b-ab62-4b5deda8afed None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.009294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.009923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.010194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.010474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.020209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.021727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.039227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:12.040514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:12.041506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:12.042384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:12.042384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.042384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:12.042384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 251/509] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:12.042598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b123654-8614-4c08-93b6-db91184ce676 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.043040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.063668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.076937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.077536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.077811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.078076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.078725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:12.079018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.080388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.088100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 259/510] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:12.088906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.144993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf583-265a-401b-855a-3b3e2051cc90 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.178724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.179319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.179587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.179863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.188716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42558d-5492-4983-8d2f-d364fa407e75 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.223111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.224469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.224851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.225162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.232406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.233915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.246526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:12.246933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:12.247877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:15:12.248780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:12.249694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.249775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 260/511] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:12.249775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.265941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.273482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.345615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 252/512] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:12.346460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.356918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba4335-f8c8-4b04-84bb-f703dc87518a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.390391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.391017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.391311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.391582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.402117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.403582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:12.418101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:12.419038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:12.419881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:12.419881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.419881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:12.419881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 261/513] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:12.420096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.446629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e3afb-c4be-4144-aedf-957d80ec5d79 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.463094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.479633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.480233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.480511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.480778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.481441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:12.481805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.483166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.494161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.495054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 253/514] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:12.495843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.519087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefa653-0e46-4d7f-90ef-6a2bff48b4f6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.553901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.554535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.554812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.555084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.565323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.566771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:12.581206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:12.582285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:12.583680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:12.583680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.583680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:12.583680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 262/515] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:12.584046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.599953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec15e4-86e5-4c75-b1fe-0ff4992c67a5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.635687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.636289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.636566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.636835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.637582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:12.637879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.639318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.656446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 254/516] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:12.657256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.685554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd2c6b-c665-4a96-8b86-31dd900e7f6e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.692925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.710633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.719104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.719727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.720000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.720268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.731154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.732606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:12.747175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:12.751046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:12.751985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:12.751985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:12.751985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:12.751985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 263/517] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:12.751985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.761023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c611f1-02d9-439f-96e2-c58edfff269a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.794273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.794814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.795091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.795367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.796002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:12.796306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.797706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.813399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 255/518] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:12.813399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.849083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ff8da-49c4-4694-ada7-10ca228a9e3f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.883435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.884052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.884326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.884592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.891405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.892830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:12.904548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:12.905453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:12.906384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 264/519] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:12.906863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.906935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:12.915772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ac7c9-d872-4179-ae37-042983b389ca None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:12.945785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.949951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:12.950555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:12.950830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:12.951099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:12.951723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:12.952019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:12.953422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:12.965005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 256/520] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:12.966130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.009088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82757445-74d0-4cc5-9795-e423d14249b8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.042868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.043476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.043756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.044028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.065756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f3b8a-9a68-4ad7-b029-83b784784118 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.100266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.100884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.101183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.101453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.102152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:13.102522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.103893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.110846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.112314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.115829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 257/521] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:13.116604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.134151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.151662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.210785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 265/522] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:13.211642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.218219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dda93e-0da2-4cb5-8fec-bf1088bc5054 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.250935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.251530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.251809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.252084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.252713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.254186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.308603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726cdee-b429-4161-9cad-e31e4b48aec3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.338765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 258/523] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:15:13 2022] POST /identity/v3/users => generated 247 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:13.339573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.342491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.343110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.343388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.343663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.344396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:13.344705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.346242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.366790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 266/524] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:13.367696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.373360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.431305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.464474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24418813-7e31-4f57-9086-c631d2fe2124 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.494727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401f1a9-e544-4b12-b8c4-293bccbcd3bb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.506674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.506674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.507574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.507834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.517827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.519286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:13.532449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:13.533448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 259/525] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:13.534402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.540113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.540914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.541162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.541438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.549432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.554787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:13.566920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:13.568187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 267/526] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:13.569235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.634692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.651153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1424-938b-4d03-99c8-210917fe71c8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.677819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955d1e0-d41b-41eb-9b38-105411b14f9f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.681916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.687131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.687759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.688040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.688509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.699650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.701163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.715684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:13.716009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:13.717221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:13.718103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:13.718103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:13.718103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:13.718279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.718359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 260/527] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:13.718359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.721617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.721928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.722690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=adc1b4c4a2de4d97bb8019c9efc05570) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:13.723011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.725131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.741130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 268/528] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/role_assignments?user.id=adc1b4c4a2de4d97bb8019c9efc05570&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:13.741130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.867469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee058f-8751-4825-9ec6-db76cbcfe12c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.894934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2dce6-5a23-4b5e-8b23-e6a71d3a43e1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:13.916080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.916841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.918715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.918715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.918715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:13.919041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.920916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.929987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:13.945072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:13.945951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:13.947716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 261/529] 10.222.0.42 () {56 vars in 1139 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:13.947716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.948541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:13.949062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:13.949964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:13.950828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:13.954790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:13.972077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 269/530] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:13.974125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:13.998123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:14.059869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817972e-89d8-4e19-ba25-ddcc45689183 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.094111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5802ae-3534-4111-8819-10fdd535ee92 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.097149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.097757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.098031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.098296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.099028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:14.099327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.100900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.113718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 262/531] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:14.115028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.140278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.140871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.141158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.141425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.198487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:14.220800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.223467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.246549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47822d20-2c1d-4c0a-9eb0-b23c25a594fe None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.248831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:14.297673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.298630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.299019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.299777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.300983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=55965a77a0fd41ce8007cea14d081007) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:14.301411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.303403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.311780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 263/532] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/role_assignments?user.id=55965a77a0fd41ce8007cea14d081007&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:14.312834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.352287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 270/533] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:14.359072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.436494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391e9d3-5132-4f7d-af57-6e3bb899a717 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.461466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:14.475155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.475861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.476069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.476346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.483946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.485513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.487502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb889d0-9168-4d8c-9981-817ee2afceeb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:14.499444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:14.500366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:14.501253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 264/534] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:14.509001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.560875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.562803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.564707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.565191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.569714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:14.581128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.583794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:14.601314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:14.602428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:14.603287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:14.603287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:14.603287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:14.604637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 271/535] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:14.617468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.763058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bc37d-b9cb-47ae-ad02-d0c8c2d2f7d8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.789208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772603b-2e09-4a76-b73e-8fcae21ef838 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:14.827779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.828058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:14.828783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.828984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:14.829276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.829549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:14.829727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.830109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:14.830563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:14.830965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.831275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:14.831744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:14.832608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.834073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:14.834814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:14.847801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 272/536] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:14.855373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.861462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 265/537] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:14.867891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:14.997370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94968f76-01ae-4df3-abc0-e50acd18acf5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:15.038850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:15.039404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:15.039683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:15.039961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:15.057866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:15.059342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:15.073442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:15.084469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:15.087912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:15.088746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 273/538] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:15.098164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:15.112597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b8191-2468-487d-9d8e-1db45db419a3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:15.169848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:15.178352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:15.179061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:15.179360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:15.179713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:15.180377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=28785166e8994ba18959790e0e5fd3d5) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:15.180886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:15.182412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:15.198184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 266/539] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:14 2022] GET /identity/v3/role_assignments?user.id=28785166e8994ba18959790e0e5fd3d5&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:15.224181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f8b0b-5829-43ba-8dc2-9728b28359be None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:15.268615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:15.269641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:15.270119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:15.270765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:15.271855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:15.272401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:15.278347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fb20f-dc74-4cf6-8cda-f91b677b5031 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:15.298423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 274/540] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:15.304580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:15.452440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55232ca-8d88-4705-947c-f3178d8ea9f5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:15.496070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:15.496725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:15.496980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:15.497243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:15.497967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=204544f0b58042c7957186df4bdd3949) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:15.498279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:15.499716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281a4ab-85dd-4889-8bc8-b5f42dc1b2c7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:15.510603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 267/541] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:15 2022] GET /identity/v3/role_assignments?user.id=204544f0b58042c7957186df4bdd3949&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:16.276818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f61b45-9bc6-4777-9c12-5dbe985e2ad4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.277353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f61b45-9bc6-4777-9c12-5dbe985e2ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.277429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f61b45-9bc6-4777-9c12-5dbe985e2ad4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.278996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 275/542] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:16.284359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.284594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.284913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.289438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbc23c-946a-4247-9e26-ac8880c133cf None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.289717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbc23c-946a-4247-9e26-ac8880c133cf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.289993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbc23c-946a-4247-9e26-ac8880c133cf None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.291508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 276/543] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:16.296142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.296419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.296696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.321433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:16.322900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:16.328680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:16.329878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:16.345110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:16.423219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 277/544] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:15:16.439815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 268/545] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:16.739661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:16.798793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:16.813872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:16.888660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe11c59-0008-4848-bcf8-ca6184f3c49c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:16.934044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a7502-adbd-4dfb-b8fd-c5c3800945e3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:16.949004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc5714-8a6f-49d4-b88c-77345fef77f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:16.949652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5714-8a6f-49d4-b88c-77345fef77f8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.949929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5714-8a6f-49d4-b88c-77345fef77f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.950202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5714-8a6f-49d4-b88c-77345fef77f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.965384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc5714-8a6f-49d4-b88c-77345fef77f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-57bc5714-8a6f-49d4-b88c-77345fef77f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:16.972591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:16.973780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:16.974856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 278/546] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:16.975524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb29a04-2df3-4439-a528-8776ab9607fe None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.975599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb29a04-2df3-4439-a528-8776ab9607fe None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.980392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb29a04-2df3-4439-a528-8776ab9607fe None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.980392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 279/547] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:16.981747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb29a04-2df3-4439-a528-8776ab9607fe None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:16.983536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e45f5-6a3c-4281-90e9-648c1ce15a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:16.984372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e45f5-6a3c-4281-90e9-648c1ce15a7a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:16.985003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e45f5-6a3c-4281-90e9-648c1ce15a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:16.985289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e45f5-6a3c-4281-90e9-648c1ce15a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:16.997014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e45f5-6a3c-4281-90e9-648c1ce15a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3d7e45f5-6a3c-4281-90e9-648c1ce15a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:17.002277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:17.003241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:17.004045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 269/548] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:17.005892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.005892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.006209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.016892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.026455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:17.028626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:17.098519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb29a04-2df3-4439-a528-8776ab9607fe None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.100965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.106188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 270/549] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:17.107272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.135607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.136218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.136494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.136763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.137581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:17.137759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.139167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.147823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 280/550] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:17.153347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.221632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05794a-8ef8-44da-bf0b-0aeb2cdf054a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.255357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f7048-6ee9-475a-b8d0-3c8e99ee7f3a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.282795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.283761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.284278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.284767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.285848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:17.286544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.288184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.290123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.300884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.300884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.300884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.300884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.305719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.306924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.309561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 271/551] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:17.314765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:17.322390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:17.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:17.324408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:17.324408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:17.324408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:17.324408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 281/552] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:17.332908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.390276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.441648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c2672-25cf-4912-8dd8-5fdc75785f7b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.494177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.494803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.495081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.495353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.508469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.510581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:17.530882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:17.531991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:17.532885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:17.532885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:17.532885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:17.533694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d13368-b0bb-4853-9ba7-88bc05febb47 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.536952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 272/553] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:17.537892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.576521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.577517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.578141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.578630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.579473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:17.579973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.581575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.589414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.597585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 282/554] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:17.599009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.666686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e602c66-93dd-44ba-9476-caf9616cb7d6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.692368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.717099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1895e0-5d35-4c32-a7aa-bbc2993efd2c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.719407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.721086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.721587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.722064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.739360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:17.744142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:17.745274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:17.746122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 273/555] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:17.746800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.759667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.760636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.761127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.761396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.762059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:17.762487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.763848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750100-004c-477c-bfc2-e227ac239aef None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.776952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 283/556] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:17.778111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523bbbb-fabe-4c9f-9016-98cffd030572 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.778392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523bbbb-fabe-4c9f-9016-98cffd030572 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.778664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523bbbb-fabe-4c9f-9016-98cffd030572 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.783947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 284/557] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:17.783947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523bbbb-fabe-4c9f-9016-98cffd030572 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.855468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeda790-dc7d-492e-94a6-6c8002795dad None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.865772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.890727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.891364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.891643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.891918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.893099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523bbbb-fabe-4c9f-9016-98cffd030572 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:17.899587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.901068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:17.913235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:17.914680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 274/558] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:17.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:17.933784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:17.934319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.934641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.934908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.935595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:17.935895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:17.937302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38be87-e18b-4c75-bc20-8de0112f7c0c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:17.942859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:17.947062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 285/559] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:17.948089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:17.948359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:17.948627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:17.966511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:17.967741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:18.029183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6d7af-c18b-4f90-a048-200429904d1b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.048897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 286/560] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:18.048897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.079210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.079971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.080525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.080801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.092330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.093806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:18.109225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:18.110220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:18.111145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 275/561] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:15:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:18.111723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.153953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.169285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bce30-195f-4742-b37f-f210158f5d77 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.198755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.204553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.205193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.205476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.205748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.206478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:18.206788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.208264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.231206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23fb4-5787-45ec-95c5-4a692e2181bf None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.248908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 287/562] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:18.248908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.270347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.270973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.271263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.271517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.281698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.283135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-30999c1d-7d11-47c8-a974-76b5a8d46e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:18.299919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:18.301064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:18.302720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:18.302720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:18.302720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:18.302720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 276/563] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:18.303410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c195e-dee4-49bc-972d-f17173f0e0e4 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.303410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c195e-dee4-49bc-972d-f17173f0e0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.303713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c195e-dee4-49bc-972d-f17173f0e0e4 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.309030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 277/564] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:18.309829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c195e-dee4-49bc-972d-f17173f0e0e4 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.353372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d67a0-dfc6-4d9b-b458-2aeb762284f0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.387787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.388394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.388675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.388982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.389709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:18.390016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.391454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.407940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 288/565] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:18.408849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.409004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c195e-dee4-49bc-972d-f17173f0e0e4 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.442524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.443123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.443399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.443669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.506994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.507312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557446a1-4cf9-4e35-966a-316ee0418892 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.508461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.541236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.541845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.542126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.542401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.543055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:18.543339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.544692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fcb722-ee81-454c-bc6c-5a77918fbcf9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.556174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 289/566] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:18.557469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.557757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.558031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.575807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:18.577080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:18.589776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.612972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.619164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 278/567] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:18.619164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.641679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.643751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 290/568] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:18.647472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.683971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.728877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6c447-39ec-4c83-87d8-ce760f52878d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.750604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce2a8a-c5b6-40ae-81b6-929dff06b925 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.768837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.769424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.769704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.770034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.780940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:18.785638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:18.786667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:18.787707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:18.790789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.791615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.791833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 279/569] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:18.792201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.792622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.792735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.805900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.807551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:18.824955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:18.826099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:18.827200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:18.827200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:18.827200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:18.828756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 291/570] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:18.829684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.840167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.899908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622c322-5a7f-4eeb-80fd-656d1ec28638 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.920547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:18.934299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4df9cf-160b-4099-ac52-97998e23a524 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:18.936971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.937604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.937885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.938162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.946015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.947735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:18.959841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:18.960825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 280/571] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:18.961850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:18.975831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:18.976464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:18.976744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:18.977042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:18.986996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:18.988443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:19.003175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:19.004304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:19.005643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:19.005643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:19.005643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:19.005643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 292/572] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:19.005643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.063110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.065868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6d9a-a4ef-41d4-971f-4a448601352c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.103013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c65956-03b5-47eb-b32a-09cdbbe6d453 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.103232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.103718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.104000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.104277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.104970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.105279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.106674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.115171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 281/573] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.115790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.138980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.139580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.139880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.140156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.140797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.141130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.142498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.151062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.154247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 293/574] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.155033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.216290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e839da4-7a8d-4aed-b029-d230d5ee33fb None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.250685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.251567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.251844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.252127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.252683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fcff8-01f1-40f7-90a5-b66d0c058951 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.262261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:19.267407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:19.268388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:19.269258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:19.270198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 282/575] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:19.271166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.288363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.288994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.289274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.289548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.290265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.290566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.291978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.294887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.310134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 294/576] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.311628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.370261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.370878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04466c-ee33-495d-815b-a1504078e3c2 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.404472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.405200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.405484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.405762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.406396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.406699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.408070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.411189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ca91b-3356-48f3-9c79-4ad08f88dc78 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.419515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 283/577] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.420316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.445458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.446093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.446372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.446644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.457686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.459174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:19.478285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:19.479241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:19.480133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:19.480133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:19.480133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:19.480133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 295/578] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:19.481428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.509950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.528252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4567f-11f0-4df0-a982-9818322a47ea None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.562355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.562977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.563258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.563534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.570772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.572530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.582979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30186955-82d0-4587-8c7d-01eecfec28b6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:19.583615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:19.584645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:19.585580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:19.589003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 284/579] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:19.590004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.593387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.616520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.617166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.617440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.617708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.618496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.618668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.620042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.628349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 296/580] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.629097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.691040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c2644-fc18-4e6a-bd61-2b9761833e94 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.724759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.725415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.725698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.725979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.728588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d8617-9ad3-4151-afc6-4eb4d94d225c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.735590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.737098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.738746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:19.752220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:19.753317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:19.754308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:19.754308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:19.754308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:19.754308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 285/581] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:19.754308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.764433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.765082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.765359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.765625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.774005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.775447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:19.787212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:19.788128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 297/582] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:19.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.822764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:19.867601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df572e23-a8c0-40f0-9e6d-37ab00c7d3e2 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.899308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7090505-9ab5-4639-a539-1c72ca5a4058 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:19.901319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.901924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.902204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.902478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.903139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.903441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.904833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.917291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 286/583] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.918090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.934366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:19.934973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:19.935248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:19.935514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:19.936200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:19.936500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:19.937943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:19.950209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 298/584] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:19.950825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:19.977185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.021719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363f6b7-e79d-4951-8915-93e0ac147b1a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.041508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.051273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755919d-472e-4512-a674-31ad8aafb5bb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.060402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.061109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.061387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.061661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.081654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.083890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:20.097638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:20.099387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:20.100254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:20.100254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:20.100254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:20.100254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 287/585] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:20.100254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.105372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.105987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.106264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.106530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.107160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:20.107459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.108822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.120238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 299/586] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:20.121065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.210574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb142e5-a276-4990-aaee-8d58ebf72e60 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.217807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.218576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18770ab4-6fe2-4134-870a-19836afa292c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.244095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.244609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.244902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.245180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.245854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:20.246160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.247569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.251690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.252317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.252597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.252885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] PATH_INFO: `/v3/users/glance` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.258906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 288/587] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:20.259737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.260631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.262104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:20.274050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:20.275035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 300/588] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:20.276013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.294801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.360067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24896ae6-55d2-4ffc-ada6-6d4692941939 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.377917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665425-ae0f-4956-8618-fa79bbf886fd None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.391936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.392594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.392881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.393179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/adc1b4c4a2de4d97bb8019c9efc05570/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.412201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=adc1b4c4a2de4d97bb8019c9efc05570, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.412512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.413169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.413451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.413724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.414372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:20.414548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.414677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.416043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.428710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 301/589] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:20.429584 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.497585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 289/590] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:20 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/adc1b4c4a2de4d97bb8019c9efc05570/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:20.498003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.498344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.521932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.537810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984c602-f862-4d7f-9b8a-76b68706de96 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.575220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.575867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.576143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.576409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.584907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.586397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:20.599123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:20.602525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:20.603473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:20.605612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 302/591] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:20.606650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.621060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcfe3e3-1284-418e-bf83-430d578be6f3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.662569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.663185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.663451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.663721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.664441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=1e3ba7be5b48443697ad07dc4ffb04fe) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:20.664751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.666230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.677396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 290/592] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/role_assignments?user.id=1e3ba7be5b48443697ad07dc4ffb04fe&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:20.678240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.751658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90247ce3-1380-447b-a153-cad9cfe97c4a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.781411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.791283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:20.797491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.798144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.798421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.798750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.799494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:20.799845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.802034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.806618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ac66-e48c-4461-9abd-749e40f839b7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:20.830969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 303/593] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:20.832012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.847544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:20.848662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:20.849177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:20.849684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:20.859115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:20.861136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:20.877293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:20.878305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:15:20.879248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:20.883779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 291/594] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:20.885089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:20.993701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab6cd1-8138-4df1-be5f-99a2ba6651ec None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.029372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.029978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.030259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.030525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.031207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:21.031506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.032937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.043354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52dfa4c-4226-42ab-9ae7-91e69e314c4b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.050145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 304/595] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:21.050145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.092880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.094013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.094567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.095058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.102975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.107232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.110047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.111941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:21.126837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:21.127807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:21.128709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:21.128709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:21.128709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:21.130894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 292/596] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:21.131865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.161163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52987475-7639-4f3e-b6a0-aee311db53ae None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.197270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.197882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.198157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.198424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.199213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:21.199435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.200870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.219003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 305/597] 10.222.0.42 () {56 vars in 1139 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:21.219959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.250610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b4692-74c5-4e85-bde1-60bea9f528ea None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.290238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.291370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.291938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.292419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/55965a77a0fd41ce8007cea14d081007/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.309497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=55965a77a0fd41ce8007cea14d081007, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.311472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.323195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce1617-d80c-4cd4-b6b7-f01b059b4e9c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.333315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.357815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.358472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.358758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.359020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.359652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:21.359950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.361351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.376922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 306/598] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:21.376922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.389633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 293/599] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:21 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/55965a77a0fd41ce8007cea14d081007/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:21.397134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.490233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.504107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db844577-4ccf-4ec1-b059-4f768d20d751 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.554875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47700dfa-7d01-4af6-a9ef-31cc92ca142b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.578699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.579359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.579642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.579916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.596145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.597224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.597720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.598412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.603787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.605332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.607744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.609783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:21.632175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:21.636937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:21.637868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:21.637868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:21.637868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:21.637868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 307/600] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:21.639592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:21.640478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:21.641454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:21.642730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.645714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 294/601] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:21.646674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.671056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.764792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec64f-3fb9-468c-87bc-bdea3e88bfb4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.814094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.814623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.814956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.815237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.815878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:21.816180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.818678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.831573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 308/602] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:21.838250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:21.863089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327f982-5e0e-4075-b651-c4802eec8aa1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.865716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.928996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:21.960018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:21.960634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:21.960938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:21.961218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:21.961939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:21.962289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:21.963730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:21.966750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eafc4-4633-48fe-961c-90d90a1ca0ca None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:21.980241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 295/603] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:21.985698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:22.052945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:22.053599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.053947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.054154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.069833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:22.071298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:22.089976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:22.090957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:22.092470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 309/604] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:22.096692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:22.183826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed41855-8c8f-4033-88c2-4c337beb7f6a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:22.230333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda72f4a-37fa-4729-b6ae-b37f80bea154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:22.231537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda72f4a-37fa-4729-b6ae-b37f80bea154 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.232038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda72f4a-37fa-4729-b6ae-b37f80bea154 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.232641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda72f4a-37fa-4729-b6ae-b37f80bea154 None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/28785166e8994ba18959790e0e5fd3d5/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.262026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda72f4a-37fa-4729-b6ae-b37f80bea154 None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=28785166e8994ba18959790e0e5fd3d5, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:22.263648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda72f4a-37fa-4729-b6ae-b37f80bea154 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:22.292666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd32de-8858-4f90-9faa-7a303da85f79 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:22.330924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:22.331602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.331980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.332367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.334292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:22.334699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:22.336282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:22.345531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:22.350326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 310/605] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:22.355822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:22.397818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 296/606] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:21 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/28785166e8994ba18959790e0e5fd3d5/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:15:22.471361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153cb92-aaa1-4efd-b3d9-63ddf9135d11 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:22.525667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba2b3d-e439-4e96-b0c7-3f7ed16c6abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:22.527207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba2b3d-e439-4e96-b0c7-3f7ed16c6abc None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.527699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba2b3d-e439-4e96-b0c7-3f7ed16c6abc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.528182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba2b3d-e439-4e96-b0c7-3f7ed16c6abc None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/204544f0b58042c7957186df4bdd3949/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.550094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba2b3d-e439-4e96-b0c7-3f7ed16c6abc None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=204544f0b58042c7957186df4bdd3949, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:22.552496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba2b3d-e439-4e96-b0c7-3f7ed16c6abc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:22.613399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 311/607] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:22 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/204544f0b58042c7957186df4bdd3949/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:22.974617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144c73-5b40-4862-81fb-55e502c7d48f None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.974806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144c73-5b40-4862-81fb-55e502c7d48f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.975179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af144c73-5b40-4862-81fb-55e502c7d48f None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.976955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 297/608] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:22.982872 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.983198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.983433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.002547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:23.003759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:23.073847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:23.077243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4deb90-b6a3-474f-abb2-4bc7edb98ece None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.077618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4deb90-b6a3-474f-abb2-4bc7edb98ece None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.082416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4deb90-b6a3-474f-abb2-4bc7edb98ece None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.082416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 298/609] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:15:23.085788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.086072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.086351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.106369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 312/610] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:23.112070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:23.113337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:23.128972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:23.228084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 299/611] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:15:23.491020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:23.594023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:23.623072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19bee5-fdbb-4c9c-8c67-4089abbede29 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:23.681357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:23.681999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.682277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.682548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.704511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:23.709637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:23.710669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:23.711542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:23.719066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 313/612] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:23.720924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:23.777531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629919-a42c-44f3-9f26-ca8650c6a712 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:23.793652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:23.839664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:23.840663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.841165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.841673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.852994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:23.858435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:23.859370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:23.860205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:23.862809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 300/613] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:23.868947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:23.872562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:23.877196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd0a08-b81d-451a-a32e-a082e2f1c8dd None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:23.915644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32986d-4190-454b-b62b-735460552175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:23.916428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32986d-4190-454b-b62b-735460552175 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.920458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32986d-4190-454b-b62b-735460552175 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.920458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32986d-4190-454b-b62b-735460552175 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.920458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32986d-4190-454b-b62b-735460552175 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:23.920458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32986d-4190-454b-b62b-735460552175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:23.920458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32986d-4190-454b-b62b-735460552175 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:23.932693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 314/614] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:23.934039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32986d-4190-454b-b62b-735460552175 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.041683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32986d-4190-454b-b62b-735460552175 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.049180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:24.071288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47cc4c-330f-4a93-a5fe-d08548b3d46c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.085617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.086265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.086547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.086822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.106404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.107897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.120887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.123208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.124163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.124613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.127431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:24.127952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.130378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:24.144339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:24.145665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:24.146545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:24.146545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:24.146545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:24.146727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 315/615] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:24.152393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 301/616] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:24.152524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.156436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.223888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:24.266113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51f753-abd8-436d-8b52-217279acb711 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.306385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.306943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.307227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.307503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.320628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.322414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:24.338486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:24.339449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:24.340331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:24.340331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:24.340331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:24.340331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 302/617] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:24.345680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.449113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c8db2-7abd-491c-8a07-71008019f6c1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.453239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01994558-8adb-4b43-a6cd-e690c04508c3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.475180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:24.483605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.484228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.484511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.484784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.485464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:24.485767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.487174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18363c1e-64b8-44f0-8810-39c3af6da4d0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.495872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.496502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.496831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.497591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.498529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:24.498847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.500270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28cd46e-bdcb-429a-b22e-2eb19942997a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.500555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 316/618] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:24.501927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bb629-6e5c-4a2a-bab3-f13a0979fd1d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.502210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bb629-6e5c-4a2a-bab3-f13a0979fd1d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.502478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350bb629-6e5c-4a2a-bab3-f13a0979fd1d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.508673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 317/619] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:24.509534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bb629-6e5c-4a2a-bab3-f13a0979fd1d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.515529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 303/620] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:24.521660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.522007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.522296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.541277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:24.546723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:24.547920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:24.628051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 304/621] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:24.629320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.696608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350bb629-6e5c-4a2a-bab3-f13a0979fd1d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.737007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4e788-5f2d-456b-9035-c39bab41ec8d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:24.753222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:24.777969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.778962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:24.780045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.780672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:24.782926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.782926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.783090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:24.783090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:24.797207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.798217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:24.800994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.802400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:24.830459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:24.831502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:24.832536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:24.835647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:24.836584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:24.837614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:24.840154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 318/622] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:24.841211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.842008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 305/623] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:24.847988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:24.969017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0824-c722-458c-874f-cde05b1e666c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.046902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.080670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248f2ee-79a2-401f-8904-13fef2cd08a8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.089474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.090446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.091499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.091984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.093412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:25.093932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.096287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.125687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.125687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.125687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.125687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.126116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:25.126116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.126116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.126568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 306/624] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:25.127954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.148758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 319/625] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:15:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:25.149627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.158327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.234281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142603a5-fd68-435b-98dc-421e59809031 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.250439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212acb-62ba-41ea-b0d2-779c1754c943 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.267080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d3f34-1f6e-4934-b70f-b29165d910a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.267698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d3f34-1f6e-4934-b70f-b29165d910a2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.267973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d3f34-1f6e-4934-b70f-b29165d910a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.268245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d3f34-1f6e-4934-b70f-b29165d910a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.278553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d3f34-1f6e-4934-b70f-b29165d910a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:25.280334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-811d3f34-1f6e-4934-b70f-b29165d910a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:25.283255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:25.284215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:25.285136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 307/626] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:25.286235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11c1b-aa60-4b0d-854e-a58d37f0ae4d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.286235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11c1b-aa60-4b0d-854e-a58d37f0ae4d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.286235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de11c1b-aa60-4b0d-854e-a58d37f0ae4d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.287896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 308/627] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:25.288179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.288890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.289240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.289798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.295087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de11c1b-aa60-4b0d-854e-a58d37f0ae4d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.299964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.302499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-189a257e-fa5d-4f95-aca6-357c65448b43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:25.318210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:25.319376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:25.320550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:25.320550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:25.320550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:25.320550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 320/628] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:25.322191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eac6f-672a-4073-8715-31aa25fba79f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.322467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eac6f-672a-4073-8715-31aa25fba79f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.322732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1eac6f-672a-4073-8715-31aa25fba79f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.323942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 321/629] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:25.324626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eac6f-672a-4073-8715-31aa25fba79f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.382761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.398950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de11c1b-aa60-4b0d-854e-a58d37f0ae4d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.423242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1eac6f-672a-4073-8715-31aa25fba79f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.433331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810090b-2e4d-4d26-b9ce-651cd9dd9c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.433938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810090b-2e4d-4d26-b9ce-651cd9dd9c2e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.434222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810090b-2e4d-4d26-b9ce-651cd9dd9c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.434502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810090b-2e4d-4d26-b9ce-651cd9dd9c2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.458436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.459034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.459313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.459586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.460248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:25.460550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.461979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5513e-2971-40a6-9d0e-49750bde8d6c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.475491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 322/630] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:25.475491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.475491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.475491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.491140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:25.492254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:25.504130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810090b-2e4d-4d26-b9ce-651cd9dd9c2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.506223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810090b-2e4d-4d26-b9ce-651cd9dd9c2e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.534721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.558909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 323/631] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:25.559737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.602594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.625458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 309/632] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:25.626777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.627063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.627330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.646618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:25.648126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:25.677542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfefda-368f-436b-8509-2c544da337fc None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.714074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.714791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.715071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.715345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.715980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:25.716281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.717696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.719327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 310/633] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:25.725225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.734459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 324/634] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:25.735338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.774282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.851097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f2aa4-03ba-46ca-8177-f7511c9a514f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.861778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10abab-53cb-4fa3-8466-d390561dc60b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:25.867911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:25.890169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.890799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.891082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.891358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.902042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:25.902339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.903376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.903823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.904167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.910047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.916831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:25.918414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:25.926910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:25.928396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:25.929377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:25.929377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:25.929377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:25.929377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 311/635] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:25.929377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:25.938660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:25.940151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:25.945689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:25.945689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:25.945689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:25.945689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 325/636] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:25.945689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.036604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0d3d0-4aa2-426e-9db5-f01b7078bbdf None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.038092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.050250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d5fb4-756f-4fff-ac9f-13cf6267db0f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.072648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.073308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.073585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.073855 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.080405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.081887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.086093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.086712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.086987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.087257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:26.092761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:26.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 312/637] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:26.095063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.099768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:26.105075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:26.106214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:26.107161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 326/638] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:26.107856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.125584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.204471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17b506-ee83-41f9-ae50-c31e88679b4c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.212102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2076410-b746-412f-82eb-a5e464d5eb8f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.239863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.240465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.240739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.241045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.241677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:26.241978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.243331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.245376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.245985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.246267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.246540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.247182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:26.247486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.248834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.254173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 313/639] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:26.255528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.265238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 327/640] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:26.265238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.271951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.339587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.353836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ac7f-c1e7-49f4-96e8-ea2d84938be3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.359882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeac74-f30f-4bb9-b03b-11a2892cf811 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.388529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.389195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.389473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.389745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.393487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.394083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.394359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.394627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.395364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:26.395743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.397177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.400402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:26.405125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:26.406322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:26.407224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 314/641] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:26.407924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.414904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 328/642] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:26.414904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.478473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.515695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f866788-462a-4b9c-92ec-ee4c821df9e0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.516997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ede7a-6d8d-47a0-83dc-5f2f7b882c7b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.549504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.550132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.550415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.550691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.552763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.553570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.553847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.554117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.554770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:26.555068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.556423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.560045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.561524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.564184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 315/643] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:26.564998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:26.574871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:26.576002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:26.577128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 329/644] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:26.577762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.577836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.675248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a9ffb-c4fd-490d-8d17-00c9c1aaeae1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.675562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f559ff-612f-41f7-959f-07e0832a38ad None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.701781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.709822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.709987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.711882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.712143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.712233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.712318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.712392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.712472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.712472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:26.712472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.713487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.717874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.719303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.721686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 330/645] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:26.722452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:26.732356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:26.733527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 316/646] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:26.734429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.798826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.826043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f4105-44a4-4389-bbe1-7da49a86c73d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.833428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f301b4c-66c3-4970-b830-2411ca971e9d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.861034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.861688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.861964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.862245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/1e3ba7be5b48443697ad07dc4ffb04fe/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.866442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.867056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.867332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.867652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.876384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=1e3ba7be5b48443697ad07dc4ffb04fe, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.877693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.877846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.879108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:26.893111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:26.894253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:26.896300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:26.896300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:26.896300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:26.896300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 317/647] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:26.896300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.943502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 331/648] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:26 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/1e3ba7be5b48443697ad07dc4ffb04fe/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:26.944327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.995281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.021051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e634a-b1a1-44a3-9bbd-df41b6ff2875 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.045200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.060137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.060756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.061059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.061333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.061439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6aad2e-3232-4e98-9f67-924fc58003ef None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.062060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:27.062554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.064076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.084938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 318/649] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:27.085648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.107508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.108159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.108441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.108715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.121331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.122738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:27.140415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:27.141650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:27.142599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:27.142599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:27.142599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:27.142599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 332/650] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:27.142826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.203801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564eb3-fa6b-492b-89f9-e314f30b3b57 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.243754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.247502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.248105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.248380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.248650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.249360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:27.249663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.251105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.252998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e6afe-3170-4d13-8e7a-e9f2f847d3e0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.267745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 319/651] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:27.267745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.296483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31925b-9426-470c-9894-93453580d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.297122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31925b-9426-470c-9894-93453580d2af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.297403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31925b-9426-470c-9894-93453580d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.297675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31925b-9426-470c-9894-93453580d2af None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.298312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31925b-9426-470c-9894-93453580d2af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:27.298613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31925b-9426-470c-9894-93453580d2af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.299986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31925b-9426-470c-9894-93453580d2af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.305801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.318395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 333/652] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:27.318395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31925b-9426-470c-9894-93453580d2af None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.373080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0a165-79cd-4142-9aae-16cbe7d1fd72 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.408660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.409301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.409584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.409909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.426865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31925b-9426-470c-9894-93453580d2af None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.463211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.463814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.464091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.464361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.465033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:27.465331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.466699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.482969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 334/653] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:27.482969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.486708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.488166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.496314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.529890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.596693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 320/654] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:27.596693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.597023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d8bc3-1e92-4b93-a693-7d5f78e917e4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.633449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.633977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.634244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.634515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.635228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=adc1b4c4a2de4d97bb8019c9efc05570) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:27.635531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.636966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.643032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 335/655] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/role_assignments?user.id=adc1b4c4a2de4d97bb8019c9efc05570&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:27.643653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.734849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.747222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc49145-ec83-4a54-b754-f279d7d6064f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.761061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff62f07-2de5-4bc0-8d71-35becdb37eea None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.787946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.789083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.789630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.790219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.798392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.799176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.799353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.799576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.801522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.803596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.805883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:27.809011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:27.810589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:27.819432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:27.820430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:15:27.821414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:27.823993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 321/656] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:27.824108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:27.824989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:15:27.825933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:27.826482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 336/657] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:27.827089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.831872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:27.952423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d5d14-e4dd-4a48-8527-95255e67baf6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.960126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065529d5-1f0b-44ac-a6d1-69b49cce0b7e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:27.998291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.999013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:27.999209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:27.999324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:27.999538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:27.999811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.000095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.000369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.001128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:28.001606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.003372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.010872 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.011515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.014299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.024527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 322/658] 10.222.0.42 () {56 vars in 1139 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:28.028293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:28.036291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:28.037272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:28.038181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:28.038181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:28.038181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:28.038181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 337/659] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:28.038822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.106442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.148810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb08d30-bbe0-4848-8b29-93430e49ba24 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.165656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beec347-ae6e-442d-b01e-e23124ea7427 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.188280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.188924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.189210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.189488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.207805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.209111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.209802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.210224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.211321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:28.212087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.213769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.238038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 323/660] 10.222.0.42 () {56 vars in 1137 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:28.239485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.258483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.273126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.275643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.372765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebe6a3-0705-4751-a209-f31fc46ae63b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.387787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.400514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 338/661] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:28.401770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.408696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.409530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.409868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.410251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.411176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:28.411580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.413331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.425877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 324/662] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:28.427506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.525929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.543687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff916-9bf5-4a92-9dc9-01a8307821fa None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.548017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be048f38-48bf-4ae0-95c7-6fe723bc6f78 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.586307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.587025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.587367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.587721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.590374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.591554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.591978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.592391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.599728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.601216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:28.623949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:28.624927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:28.625844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:28.625844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:28.625844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:28.625844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 325/663] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:28.626476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.655945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.692503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.694104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.753633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36bd18-17d9-4442-9d47-1b52b2a3d0cd None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.791700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.801693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.802495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.802774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.803043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.808995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 339/664] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:28.810265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.811206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.813145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:28.825493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:28.826634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:28.827760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:28.832903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 326/665] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:28.832903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:28.939320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd11412-887f-40c1-91fb-f7717521f360 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.940523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:28.961697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfcbdb-66e9-4dcb-b123-e59be27965da None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:28.979807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:28.980729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.981274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.981759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.982667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:28.983173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:28.984737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:28.997175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 340/666] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:28.998412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.005758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.006395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.006665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.006932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.021506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.022958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:29.043525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:29.044499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:29.045476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:29.045476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:29.045476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:29.045476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 327/667] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:29.045725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.065571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.134149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d0d94-cb5f-40e9-b63c-2b7552592d1c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.170801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.171547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.171920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.172290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.173111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:29.173510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.175022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.177744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568c6a0-cbc6-4066-b330-07a755fa91ed None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.188689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 341/668] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:29.189695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.219623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.220531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.220810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.221127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.227885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.229371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.231057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:29.241544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:29.247718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 328/669] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:29.248642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.298865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4926418-d405-481d-a1a3-240c84f58c74 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.309078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.334910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.335512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.335792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.336065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.336695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:29.337028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.338620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19b6c1-b6a3-4ea3-8c51-a80fbdd90d76 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.350488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 342/670] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:29.351086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12f990-7bf1-4540-96f0-862e56b81cb1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.355968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523009-11c8-45cd-95ee-513da18405d9 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.356257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523009-11c8-45cd-95ee-513da18405d9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.356532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96523009-11c8-45cd-95ee-513da18405d9 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.357948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 343/671] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:29.358565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523009-11c8-45cd-95ee-513da18405d9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.391117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.392309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.392581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.392842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.393581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=55965a77a0fd41ce8007cea14d081007) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:29.393938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.395413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.401399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 329/672] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/role_assignments?user.id=55965a77a0fd41ce8007cea14d081007&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:29.402024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.473877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.510118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96523009-11c8-45cd-95ee-513da18405d9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.519515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7bdd6-39f6-4a56-91f5-d5c64bdd747d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.560032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.560684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.561158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.561533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.563339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.564435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.565308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.565771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.567268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:29.567769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.569721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede4dd1-fff0-4a66-8c29-367fc8dd2e70 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.570742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.572634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:29.586554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:29.587596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:29.588516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:29.591289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 344/673] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:29.592497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 330/674] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:29.593154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.593504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.593834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.594383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.617301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:29.618852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:29.651601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.714403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700da1f8-8ae4-4f18-9054-adecfffb6934 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.720919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 345/675] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:29.720919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.753006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.774471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.775148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.775429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.775698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.776395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:29.776698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.778173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.802652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 331/676] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:29.812706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.905967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef66417-614c-409b-9a9c-9611a969bf30 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.981156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.981765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.982037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.982304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.983027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=204544f0b58042c7957186df4bdd3949) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:29.983328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.984758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.999318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 346/677] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/role_assignments?user.id=204544f0b58042c7957186df4bdd3949&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:30.013316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa829ce-1d54-4bc8-9914-3416589ce501 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:30.052387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:30.063681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:30.063681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.063681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.063681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.064078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=28785166e8994ba18959790e0e5fd3d5) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:30.064078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:30.064078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6489c2-dc61-48c5-a638-9acf167d9885 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:30.076677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 332/678] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/role_assignments?user.id=28785166e8994ba18959790e0e5fd3d5&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:30.078367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0e4fa-1943-46f1-9de3-b0816575b722 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.078815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0e4fa-1943-46f1-9de3-b0816575b722 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.079230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0e4fa-1943-46f1-9de3-b0816575b722 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.086883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 333/679] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:30.088770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.089247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.089725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.107140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:30.112484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:30.117050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:30.210016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:30.267197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea024a-e167-4438-91a8-977b3c46f3eb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:30.268940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 334/680] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:30.322533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:30.323147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.323438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.323715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.335186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:30.340377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:30.341359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:30.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 347/681] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:30.348265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:30.460049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22367638-1195-4777-a57c-5bc40ebac62e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:30.496498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:30.497135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.497410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.497681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.498337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:30.498637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:30.499987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:30.509108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 335/682] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:30.514980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:30.590669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:30.607149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:30.626240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c07af-ae47-4e1c-93cf-3f36dc9607d0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:30.677697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:30.678338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.678675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.678949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.686456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:30.698014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:30.699486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:30.727900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:30.729061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:30.729959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:30.729959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:30.729959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:30.729959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 348/683] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:30.735381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:30.819269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a878667-1d45-4340-8c4e-6499bc8c8d91 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:30.844970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e049a0-c0f4-41ac-a4f4-ac84946d33bf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:30.895402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:30.896016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.896298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.896577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.899577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:30.900186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:30.900466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:30.900741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:30.901420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:30.901728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:30.903236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:30.921217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:30.924827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:30.929929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:30.930888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:30.931734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:30.938518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 349/684] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:30.942875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 336/685] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:30.945543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:30.946854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:31.076558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:31.081112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e1f93-3dfe-4a0a-a6a5-f1d714ab461b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:31.130781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:31.131331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:31.131609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:31.131882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:31.148519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c578d-3e8f-4126-a5dc-dbfec215fbb2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:31.150109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:31.151586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:31.173154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:31.174176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:31.175124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 337/686] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:31.180208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:31.255028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:31.255670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:31.255953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:31.256226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:31.256928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:31.257236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:31.258650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:31.283430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 350/687] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:31.294923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:31.401280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:31.467901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b5c34-2513-45ba-9b9c-7f6a10788e18 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:31.481809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d856d-478c-40e7-9222-cd6bcdace689 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:31.540073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:31.541667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:31.542500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:31.542777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:31.543511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:31.543817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:31.545280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:31.551182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:31.551971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:31.552256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:31.552531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:31.565129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:31.566705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:31.571287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 338/688] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Oct 4 10:15:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:31.578350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:31.588802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:31.589821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:31.590725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:31.590725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:31.590725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:31.593882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 351/689] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:31.600686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:31.627097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:31.784442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50adf19-4d9b-4fb0-a768-6ee425eefb8d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:31.803720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb7397-b05f-497e-a7c6-b7e90f2f666f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:31.814150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:31.841535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:31.842243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:31.842523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:31.842787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:31.843417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:31.843715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:31.845238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:31.859550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 352/690] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:31.864066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f3862-636b-48e7-930e-c00dfd41674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:31.864688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3862-636b-48e7-930e-c00dfd41674f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:31.864998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3862-636b-48e7-930e-c00dfd41674f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:31.865348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:31.865471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3862-636b-48e7-930e-c00dfd41674f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:31.951494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3862-636b-48e7-930e-c00dfd41674f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:31.953288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3862-636b-48e7-930e-c00dfd41674f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:31.976562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.003697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f975c3-f1fc-4bd8-9f5a-3113b508f119 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:32.044673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:32.045364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.045648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.045925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.054016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:32.056229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-165d6a16-5161-45be-a50c-27768b334eb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:32.070567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:32.071622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:32.072883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 353/691] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:15:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:32.073781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c585cf-5ada-4c46-9d10-922162b2418a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.073874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c585cf-5ada-4c46-9d10-922162b2418a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.074142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c585cf-5ada-4c46-9d10-922162b2418a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.080323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 354/692] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:32.081150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c585cf-5ada-4c46-9d10-922162b2418a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.102545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 339/693] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 10:15:32.104037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.104320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.104597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.119974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.130765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:32.131490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:32.224958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 340/694] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:32.226119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.229217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c585cf-5ada-4c46-9d10-922162b2418a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:32.273245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:32.274726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.275312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.275731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.277384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:32.277857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:32.279549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:32.302401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 355/695] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:32.310392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.322833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.395290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1deea-3eb5-491b-849e-d9455557341b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:32.437389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:32.438084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.438362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.438676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.451521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:32.453496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:32.471452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:32.475576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:32.476485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:32.476485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:32.476485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:32.476485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 341/696] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:32.479459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.534095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.600309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e2fa0-7ea9-4bed-a437-f98c2fd83357 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:32.680155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:32.681061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.683307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.687909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.721561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.734846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5b6e4-8ef3-43d6-90e9-f48458f0115b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:32.796128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:32.798247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.798728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.799472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.801922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:32.802515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:32.804512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:32.815093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:32.816640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:32.834439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 342/697] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:32.835994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.933240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.951299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 356/698] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:15:32.952215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.981188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67b3cd-8743-4de4-8dda-262319031660 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.017997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.018638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.018911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.019180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.020332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.021730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3a4fa2e1-ed7e-4503-9d46-3018f5640f88 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:33.032667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:33.033811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.034681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 343/699] 10.222.0.42 () {56 vars in 1129 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:33.036719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c1d43-0deb-416b-b6fc-4f5592dfe35d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.036719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c1d43-0deb-416b-b6fc-4f5592dfe35d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.036719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c1d43-0deb-416b-b6fc-4f5592dfe35d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.038066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 344/700] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:33.040351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533679c-41c1-4679-9dc7-da116b3f9adf None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.040643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533679c-41c1-4679-9dc7-da116b3f9adf None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.041109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533679c-41c1-4679-9dc7-da116b3f9adf None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.042515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 345/701] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:33.043521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533679c-41c1-4679-9dc7-da116b3f9adf None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.076218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237ecba-92a0-46cf-9933-d9e7527a12ce None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.080786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.111530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.112185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.112458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.112725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.119596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.121072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:33.131867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:33.132966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 357/702] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:33.134131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.163221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533679c-41c1-4679-9dc7-da116b3f9adf None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.180999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.198105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.198658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.198933 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.199201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.209174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.210616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-dad99507-c291-4fe7-b695-4652e560fbae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:33.226263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:33.227255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:33.228163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:33.228163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:33.228163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.228163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 346/703] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:33.228526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.228792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.229107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.241088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9d415-82a4-4e6f-b322-3d70b4c80a3b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.246687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:33.247868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:33.274506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.275123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.275399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.275670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.276294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:33.276593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.277994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133db397-00be-4d36-92a1-9da32c685279 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.282418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 358/704] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:33.283573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.283854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.284127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.302708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:33.303885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:33.313784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 347/705] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:33.314634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.328797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.377486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 359/706] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:33.385728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.440454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.445297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253900f1-373f-4a9d-9cc0-6cb5a709df06 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.482343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.483771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.484313 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.484781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.486091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:33.486614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.488825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.502828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 348/707] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:33.503721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.517008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e6276-bef7-4602-ae63-aa152a02150e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.563865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.565527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.566023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.566924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.568373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:33.568944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.571022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.571861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.586600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 360/708] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:33.587638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.616658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954a6c5-3fc9-4447-b3a7-561077ddf8ce None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.656642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.657282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.657566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.657841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.658481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:33.658783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.660208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.666220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 349/709] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:33.667002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.699890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.703996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54282b-4c66-48bb-a451-385213dbccac None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.749101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.750027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.750493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.750946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.751888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=1e3ba7be5b48443697ad07dc4ffb04fe) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:33.752387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.754860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.761595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 361/710] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/role_assignments?user.id=1e3ba7be5b48443697ad07dc4ffb04fe&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:33.762881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.783257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6f399-f888-4204-b25f-1010576e908e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.815629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:33.825311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.825966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.826240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.826510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] PATH_INFO: `/v3/services/network` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.827163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.828820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:33.841735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:33.842830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:33.849441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 350/711] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:33.850494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:33.909953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d9bd6-d49d-4ca6-877c-f39e96e933d6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.955591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:33.956743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:33.957142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:33.957482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:33.967950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:33.969835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:33.973596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2508f6-f65a-43f8-b129-01a7dfc6b689 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:33.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:33.985444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 362/712] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:33.991959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:34.020088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:34.020677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.021355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.021632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.021909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.034348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.036331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:34.057369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:34.058446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:34.059553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:34.059553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:34.059553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:34.059553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 351/713] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:34.059792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:34.114537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:34.134554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7719cf-e780-4433-90c1-233a81d0d34e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:34.179135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7149aa9-1e60-442a-adb1-c3cb5f0b2625 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:34.185653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.186552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.186983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.187404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.188739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:34.189217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.191197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.199739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 363/714] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:15:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:34.200905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:34.218635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.219457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.219779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.220140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.221044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:34.221442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.223726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.237345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 352/715] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:34.243118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:34.317596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:34.335347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1456e0-6145-4117-a389-6a72f9656eb4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:34.355022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bfc53-1128-45e5-9836-4e6783739c9b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:34.357578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:34.381129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.382273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.382708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.383266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.384395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:34.385128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.387145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.392621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.393428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.393804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.394220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.395722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] RBAC: Adding query filter params (type=network) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:34.396282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.397877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.403405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 353/716] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:15:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:34.403849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 364/717] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:34.404973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:34.418495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:34.527674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2a383-608d-48d0-a3a6-fabad3a1651f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:34.600989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.601928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.602378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.602823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.603719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=adc1b4c4a2de4d97bb8019c9efc05570) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:34.604201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.605850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a565f7-c1a5-4a73-b38b-1e518efb2b27 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.620885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb3680-1310-40ef-a2ae-a7ab8a23f2fb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:34.623385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 354/718] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:34 2022] GET /identity/v3/role_assignments?user.id=adc1b4c4a2de4d97bb8019c9efc05570&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:34.651000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feff9f-23ac-4e72-9f45-91d81cc04af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:34.651736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feff9f-23ac-4e72-9f45-91d81cc04af9 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:34.652086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feff9f-23ac-4e72-9f45-91d81cc04af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:34.652434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feff9f-23ac-4e72-9f45-91d81cc04af9 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:34.653282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feff9f-23ac-4e72-9f45-91d81cc04af9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:34.654795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feff9f-23ac-4e72-9f45-91d81cc04af9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:34.763711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 365/719] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:15:34 2022] POST /identity/v3/users => generated 254 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:15:36.166988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146b191-e3e7-4ccb-8579-ae73f05cb3cf None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:36.168023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146b191-e3e7-4ccb-8579-ae73f05cb3cf None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:36.168505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146b191-e3e7-4ccb-8579-ae73f05cb3cf None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:36.170119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 355/720] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:36.180929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6359d-af14-425f-b018-1263f2514f66 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:36.181340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6359d-af14-425f-b018-1263f2514f66 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:36.181514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6359d-af14-425f-b018-1263f2514f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:36.193276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:36.208548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8bd6359d-af14-425f-b018-1263f2514f66 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:36.210166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8bd6359d-af14-425f-b018-1263f2514f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:36.306235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 366/721] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:36.511539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3e649-4915-4fbf-900b-98d24930a2b4 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:36.511850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3e649-4915-4fbf-900b-98d24930a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:36.512176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3e649-4915-4fbf-900b-98d24930a2b4 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:36.513791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 356/722] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:36.519022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:36.519303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:36.519720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:36.545755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:36.546945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:36.648073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 367/723] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:36.768671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3e649-4915-4fbf-900b-98d24930a2b4 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:36.808499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:36.902553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3e649-4915-4fbf-900b-98d24930a2b4 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:36.948249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:36.948906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:36.949196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:36.949473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:36.969783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:36.974696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:36.975657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:36.976544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 357/724] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:36.985944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:36.991137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:37.058364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:37.089268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:37.097217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0ac4b-0348-433c-86a1-bfc7035eb339 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:37.098546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aeb54-26ca-4f7a-a25b-7f618a3592f1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:37.137028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:37.137904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.138188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.138464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.139704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:37.140008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:37.141403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59199a96-09d2-4f26-bd59-4a04863771e0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:37.145961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f190d8f-b1b3-40f8-a2e9-ff407b71d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:37.147436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f190d8f-b1b3-40f8-a2e9-ff407b71d3b7 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.147954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f190d8f-b1b3-40f8-a2e9-ff407b71d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.148721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f190d8f-b1b3-40f8-a2e9-ff407b71d3b7 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.149611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f190d8f-b1b3-40f8-a2e9-ff407b71d3b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:37.155121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 368/725] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:37.155121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e83d8-a22a-41c9-be33-6cd5a6517d44 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.155121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e83d8-a22a-41c9-be33-6cd5a6517d44 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.155121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e83d8-a22a-41c9-be33-6cd5a6517d44 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.155121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 369/726] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:37.156052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e83d8-a22a-41c9-be33-6cd5a6517d44 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:37.156641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f190d8f-b1b3-40f8-a2e9-ff407b71d3b7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:37.245574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 358/727] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:15:36 2022] POST /identity/v3/services => generated 251 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:15:37.245574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44769fe-47d9-46ce-9bc0-32c56740b66b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.245574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44769fe-47d9-46ce-9bc0-32c56740b66b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.245574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44769fe-47d9-46ce-9bc0-32c56740b66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.284421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d44769fe-47d9-46ce-9bc0-32c56740b66b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:37.287438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d44769fe-47d9-46ce-9bc0-32c56740b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:37.333847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e83d8-a22a-41c9-be33-6cd5a6517d44 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:37.365588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:37.378230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:37.380437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.380437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.380437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.385380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 359/728] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:15:37.387005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4aaf6-cbcd-46a8-a906-173eb917b8a5 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.387491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4aaf6-cbcd-46a8-a906-173eb917b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.387861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4aaf6-cbcd-46a8-a906-173eb917b8a5 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.389572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 360/729] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:37.397871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:37.399669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:37.402715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.403002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.403276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.433090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:37.434520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:37.441831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:37.443010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:37.443911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:37.443911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:37.443911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:37.444507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 370/730] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:37.450414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:37.537334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 361/731] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:15:37.577956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4db18-b169-4e24-9e68-8139d300e462 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:37.621819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:37.622433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.622708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.623023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.623668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:37.623964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:37.625407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:37.639853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 371/732] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:37.645359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:37.661861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:37.672400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:37.714330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:37.791622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b45ed-7f95-4b78-a9ec-9afea72b7b00 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:37.827405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b05e6-4f32-4368-8620-c482a95e3ad0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:37.848650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:37.849914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.850215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.850492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.857935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:37.859384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:37.869160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:37.869702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.869975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.870243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.870895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:37.872233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ca8e91ee-b2cd-4635-ada6-1c4c77ee683a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:37.872659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:37.873725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 362/733] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041bc3a-f54b-494e-b7fa-ec07f2a81dc0 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:37.880504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041bc3a-f54b-494e-b7fa-ec07f2a81dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:37.882092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041bc3a-f54b-494e-b7fa-ec07f2a81dc0 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:37.882092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 363/734] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:37.882092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041bc3a-f54b-494e-b7fa-ec07f2a81dc0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:37.953732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:37.956401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 372/735] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:15:37 2022] POST /identity/v3/services => generated 240 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:15:37.958328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.001431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041bc3a-f54b-494e-b7fa-ec07f2a81dc0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.044586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fa293-f750-4891-ad1a-b5d289f63b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.045319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa293-f750-4891-ad1a-b5d289f63b1f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.045599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa293-f750-4891-ad1a-b5d289f63b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.045872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa293-f750-4891-ad1a-b5d289f63b1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.059868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fa293-f750-4891-ad1a-b5d289f63b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3b5fa293-f750-4891-ad1a-b5d289f63b1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:38.064844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:38.065860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:38.066802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 364/736] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:38.067897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.068184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.069086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.089994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:38.091769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:38.101949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.110257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f6fc4-a443-41c9-9662-930f89b4bbee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.157852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.158914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.159405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.159674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.160597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:38.161022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:38.162462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:38.175398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 365/737] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:38.176905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.182967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 373/738] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Oct 4 10:15:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:38.191635 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.257408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.309826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c960dd5-6e41-402f-a13f-fea5f57a606b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.310395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275e6fa-f82a-4d51-8726-48f0a505bfab None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.357746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.358357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.358632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.359030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.359701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:38.359991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:38.361383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:38.362231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.364060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.364589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.364900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.371464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 366/739] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:38.376504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.410080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.502879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:38.504899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:38.524387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67ff49-3e98-4390-b651-be7ced69c0e5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.560097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.581550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.582680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.583182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.583649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.598093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:38.600135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:38.620598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:38.621653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:38.622571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:38.622571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:38.622571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:38.624775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 367/740] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:38.626681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.671392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 374/741] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:15:38.672273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.733488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.780759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0dd5b-ef82-4627-911c-b82e5224de97 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.792893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184066e-d880-4ab5-9505-230944f82ff3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.818000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.818640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.818919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.819191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.830043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:38.836100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:38.837111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:38.838007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 368/742] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:38.838715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.849055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.849669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.849944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.850209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.850853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:38.851150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:38.853123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:38.868457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 375/743] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:38.869538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.876064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.954392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66bceb-d186-437c-afba-7bb791819d19 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.996777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143221cb-e815-461a-9107-b615eb2bfdd3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.997646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.998546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.998825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.999273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.011659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.013664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.014861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:39.031367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:39.032567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:39.033856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:39.033856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:39.033856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:39.033856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 369/744] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:39.033856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.038611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.039491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.040028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.040507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.041392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:39.041891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.043445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.058417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 376/745] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:39.059561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.132801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.150663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41cabf-edef-4837-bba8-5219d037d6d9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.168196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934f127-2357-47f2-8eac-a75b58738519 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.187606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.188227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.188502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.188772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.197391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.198826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.205111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.205935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.206419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.207526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.208366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:39.208876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.210474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:39.211053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:39.211960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:39.213038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 370/746] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:39.213643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.230918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 377/747] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:39.231965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.266763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.321403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed628e94-e7f0-4c9c-8472-b218f67c012b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.341271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51045a-3424-493b-994b-e599a8b93ac6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.359587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.360307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.360672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.361170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.373499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.375113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.377844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.380161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.381459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.381979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.382486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.383446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:39.383924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.385615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:39.391436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:39.397402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:39.398314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:39.398314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:39.398314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:39.398314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 371/748] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:39.398314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.415585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 378/749] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:39.416801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.514727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b72eea-1e51-4523-a8c0-dbe01a1a5b66 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.524397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.534707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcb3c6-4f12-4f8a-9d5e-8c073d14400b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.558158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.558791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.559064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.559333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.568414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.571267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.576615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.577571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.578313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.578783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.579630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:39.580128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.581714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:39.584234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:39.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:39.586041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 372/750] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:39.586528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.603117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 379/751] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:39.604510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.653245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.699117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f232fa-16fd-4afd-ac4d-ca03539af9cd None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.721226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b4808-1a1e-4925-bdfd-8c48830ceb8a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.736562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.737705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.738389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.738866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.750682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.752609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.758582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.759185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.759458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.759729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.760397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:39.760694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.762696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:39.770282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:39.771221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:39.772103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:39.772103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:39.772103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:39.774331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 373/752] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:39.775419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.776187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 380/753] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:39.786150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.802279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.901559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d0cc4-c2b1-4a26-a87a-7d84f7254cb6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.930477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c17c1d-99d7-48a0-b42a-a162b08ba068 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:39.942884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.943462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.943592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.943783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.944033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.944671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:39.945023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.946414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.960921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 381/754] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:39.960921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:39.970828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:39.971454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:39.971735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:39.972010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:39.981110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.982565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-9777052a-f2ef-4f8e-850e-8aca258a5c6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:39.996235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:39.997361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:39.998402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:40.001011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 374/755] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:40.002818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe28e6-b15c-42c0-a631-e10d1602d171 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.004532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe28e6-b15c-42c0-a631-e10d1602d171 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.005067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe28e6-b15c-42c0-a631-e10d1602d171 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.007076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 375/756] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:40.008021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe28e6-b15c-42c0-a631-e10d1602d171 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.042164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.073282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf7a5f-32dd-494b-ab67-9db8f48400d1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.112567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.113345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.113680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.114013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.114802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=1e3ba7be5b48443697ad07dc4ffb04fe) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:40.115174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.116662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.122240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 382/757] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:39 2022] GET /identity/v3/role_assignments?user.id=1e3ba7be5b48443697ad07dc4ffb04fe&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:40.122716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe28e6-b15c-42c0-a631-e10d1602d171 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.123037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.168684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.169803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.170629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.171397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.181193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.182969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-cf1db615-0620-4e00-b342-da696e9f5fbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:40.201064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:40.202253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:40.203367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:40.206562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 376/758] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:40.208971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.209405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.209818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.238958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:40.240815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:40.272736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308eff4e-f724-4a8d-87a8-d824fe44ca34 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.274126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.321825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.322786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.323163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.323677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.324580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:40.325022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.325565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 377/759] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:40.326951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.334573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.343196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.374155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 383/760] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:40.381863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.447845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b54bb6-687c-4481-a8bb-2b69910d5f61 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.493796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.494749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.495761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.496229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.497496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:40.498027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.500600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ac742-5fcb-41ea-aeb6-57967645ecb8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.517932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 378/761] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:40.520058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f0522-ebf3-40a1-ad1d-5153fe6aec39 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.520655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f0522-ebf3-40a1-ad1d-5153fe6aec39 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.521120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f0522-ebf3-40a1-ad1d-5153fe6aec39 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.522807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 379/762] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:40.528314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.529263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.529727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.553171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed7c4f-a301-4206-adc2-cd00c243b44a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.554213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:40.556783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:40.575488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.594302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.595196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.595677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.596200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.654709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 380/763] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:40.655695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.681241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.689440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.691453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.773991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62151af8-beb8-4fbe-86f2-5b49543f1f9e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.811845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 384/764] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:15:40.811845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.815891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.817500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.818005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.818478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/adc1b4c4a2de4d97bb8019c9efc05570/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.820713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=adc1b4c4a2de4d97bb8019c9efc05570, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.822752 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.897262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 381/765] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:40 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/adc1b4c4a2de4d97bb8019c9efc05570/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:40.898672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:40.913437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.921313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7399c-e5ca-4943-a381-e2f07afe26e9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:40.924290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:40.965885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:40.965885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:40.966233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:40.966449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:40.967049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:40.968476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:40.982032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:40.983499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:40.984518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 385/766] 10.222.0.42 () {56 vars in 1129 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:40.985469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.061105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b8833-b918-4a64-8e57-e9c5a4b8b083 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.107935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.109000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.109286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.109568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.124149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae95277-3fbe-4b84-9a87-06546e11885d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.124343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.125377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:41.141743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:41.142734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:41.143738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:41.143738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:41.143738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:41.143738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 382/767] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:41.143738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.166962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.167885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.168468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.168747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.169503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.170887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:41.181420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:41.182436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:41.183679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 386/768] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:40 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:41.184511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.212304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:41.218888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:41.270128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42f7a-0e48-47b1-b3a7-e58837235d8c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.311827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b404f-cb03-4cb4-b8a5-eae81b6d7bb5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.314445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.315461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.315461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.315913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.316707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.317238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.318692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.328677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 383/769] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:41.329661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.360954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.362140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.362633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.363626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.364799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.365356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.367654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.388408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 387/770] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:41.390031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.462797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01ae09-a3fc-4fa4-ba64-cd5ea4d2145b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.493281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:41.502416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:41.504353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.506319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.506319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.506319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.506319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] RBAC: Adding query filter params (name=network) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.506591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.507814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.516907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 384/771] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:41.516907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.532372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2f088-eb83-488a-b23c-b3a34e168cb8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.581811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.582774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.583239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.583697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.584685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.585194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.587536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.595165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 388/772] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:41.596154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.633247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e347301-cd04-4fff-a99f-1d2427bc0446 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.668892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.669546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.669829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.670106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.677147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.678602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:41.690330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:41.692006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:41.693127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:41.697398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 385/773] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:41.698412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.758463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:41.759294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c372c-0ea6-4294-b581-9d7f1c44890b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.812441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.813697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.814187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.814660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.815146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fce56a-7f6b-4c5b-b635-e727c9d39af9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.815745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.816826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.819628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.831030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 389/774] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:41.831030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.844762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:41.864265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.865682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.866325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.866917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.867838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] RBAC: Adding query filter params (interface=public, service_id=8f3d59efb17548969c5cecb65264acf8, region_id=RegionOne) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.868622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.871066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:41.881893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 386/775] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/endpoints?service_id=8f3d59efb17548969c5cecb65264acf8&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:41.883125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.951873 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac45a1f-a928-453c-a07f-e6ed1bc2f7a2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:41.990057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:41.990730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.991011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.991284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.992026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:41.992346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:41.993794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:42.013342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 390/776] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:42.014169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:42.031167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fa41b-4e97-4bb1-9774-1959b128d32f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:42.049662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:42.073393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:42.074110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.074402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.074676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.075434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] RBAC: Adding query filter params (interface=public, service_id=52cfa6b2cd8f4c06a6f315ca8ba6ff25, region_id=RegionOne) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:42.075750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:42.077166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:42.088159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 387/777] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/endpoints?service_id=52cfa6b2cd8f4c06a6f315ca8ba6ff25&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:42.088159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:42.122733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:42.170446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c320599-266c-4c9b-9e62-2691c37d1c9c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:42.210020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:42.210640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.210920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.211208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.211789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:42.212975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232674c-5f88-4bc6-ab0d-58957f00142f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:42.213194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:42.220095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 391/778] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:15:42 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:42.221323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:42.296462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:42.297035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.297318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.297597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.298334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=64622f7e541046d69f64a4393316abd7) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:42.298699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:42.300286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37762bd5-5781-4e3d-b12f-1bcee16d110f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:42.315276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 388/779] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:42 2022] GET /identity/v3/role_assignments?user.id=64622f7e541046d69f64a4393316abd7&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:42.353713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768f633-8475-48de-b8f6-cf6a05892f72 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.354393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768f633-8475-48de-b8f6-cf6a05892f72 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.354393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768f633-8475-48de-b8f6-cf6a05892f72 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.355866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 389/780] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:42.358414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4b6d-ec9a-4c89-b85a-11bf9da840fb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:42.362298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.362298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.362298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.382249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:42.383669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:42.408075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:42.409490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.410534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.412387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.413578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:42.415578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:42.417895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:42.426238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 392/781] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:15:42 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:42.433403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:42.481044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 390/782] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:42.784411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:42.891492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714489-418c-450d-83fc-0ec7784a97d5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:42.934888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:42.935579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.935862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.936129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.950198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:42.955006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:42.961753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:42.962619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 393/783] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:42.963315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:43.093340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d88a4d-944e-467c-a820-1ed2bee72a40 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:43.146143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:43.146689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.146972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.147248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.147939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:43.148244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:43.153826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d088fe66-edc5-45b5-9c94-fce7c019e120 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:43.163974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 391/784] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:43.169604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:43.240336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:43.243078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6696d-c711-4260-84de-5c0e4677c48c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.243364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6696d-c711-4260-84de-5c0e4677c48c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.243648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6696d-c711-4260-84de-5c0e4677c48c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.245147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 392/785] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:43.250331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.254443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.254443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.286631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:43.287850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:43.303861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d088a6c-f40b-4510-bba8-30ddf65caff3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:43.348707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:43.373549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:43.374196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.375614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.375904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.398716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:43.400244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:43.427029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 393/786] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:43.430140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:43.431367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:43.432265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:43.432265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:43.432265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:43.432265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 394/787] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:43.437075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:43.576603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0ce2-6fab-4c51-860d-1b4f76974de9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:43.631448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:43.632068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.632350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.632628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.633306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:43.633611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:43.634977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:43.648782 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 394/788] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:43.653918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:43.704431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:43.741253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:43.830973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb573-b222-4653-a350-c1dc9685f455 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:43.861294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f18ea-c30a-4c03-a78d-875a5da0552b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:43.875027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:43.875658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.875931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.876201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.885304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:43.900079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:43.904750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:43.905722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:43.906576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:43.907297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 395/789] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:43.913166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:43.930148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:43.930778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:43.931051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:43.931329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:43.947429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:43.950203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:43.967751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:43.968763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:43.969766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:43.977309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 395/790] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:43.982806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:44.017780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:44.047265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f52131-77b2-443f-ab24-554f422b6f77 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:44.103201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:44.103997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.104274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.104593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.105304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:44.105607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:44.106978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:44.117523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 396/791] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:44.122844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:44.175012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fae7a-9b54-436a-84ec-0937bbc7b6a3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:44.214030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:44.214647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.214923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.215191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.215916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:44.216244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:44.218363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:44.239549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 396/792] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Oct 4 10:15:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:44.244745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:44.276458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:44.354388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217e78-4cf9-47bc-b758-9c851568259f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:44.356625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700433fc-fc47-4ea0-9cf0-012b07cb65d3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:44.392957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:44.394040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.395365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.395839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.406979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:44.408742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:44.424973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:44.426149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:44.427050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:44.427050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:44.427050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:44.429634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 397/793] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:44.437810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:44.449655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:44.450288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.450563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.450829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.469672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:44.471123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:44.478603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:44.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:44.499405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:44.500397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:44.500397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:44.500397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:44.505500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 397/794] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:44.514833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:44.604386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:44.646937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4129a3f-a319-4fd1-b457-ec64069e09f8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:44.702674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b5c6-7ce7-409d-833d-62e749cb7459 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:44.715306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:44.715983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.716261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.716526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.717240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:44.717543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:44.721240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7749c4-c401-4f35-9bb6-ca7282fe6df2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:44.785863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:44.786550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.786827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.787099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.787756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:44.788114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:44.789522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:44.813465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 398/795] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:44.820672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:44.949091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:44.991305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 398/796] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:44.991305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f293e2-2b00-424b-a8ae-f1c64aa96110 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.991557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f293e2-2b00-424b-a8ae-f1c64aa96110 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.991766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f293e2-2b00-424b-a8ae-f1c64aa96110 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:44.993617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 399/797] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:44.998408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:44.998794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:44.999426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.019784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:45.021228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:45.025433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6b7a9-3e50-4c59-a29a-bc1dc3eebf23 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.074339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.075000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.075286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.076121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.086905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:45.088939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:45.103112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 400/798] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:45.103976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:45.105106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:45.106298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 399/799] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:15:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:45.107461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.114658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.207310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.249749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4cad-c31e-4728-a979-acf606d4f9e2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.279559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac13389-5d6f-468a-851f-8d6ddbd48be1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.289095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.289723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.289998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.290269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.296437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.297717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:45.299036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b3c59462-6681-4b6a-a655-63bc9253dbbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:45.311474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:45.312378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:45.313333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 401/800] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:45.314471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f243-2566-48fd-944d-ed7f8944d16c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.314471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f243-2566-48fd-944d-ed7f8944d16c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.314471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6f243-2566-48fd-944d-ed7f8944d16c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.320250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 402/801] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:45.320250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981c452-0542-41b8-95a3-9b29bff0fc34 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.320250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981c452-0542-41b8-95a3-9b29bff0fc34 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.320250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981c452-0542-41b8-95a3-9b29bff0fc34 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.320250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 403/802] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:45.320250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981c452-0542-41b8-95a3-9b29bff0fc34 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.328428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0af87-9119-4c16-a228-655344585951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.329085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0af87-9119-4c16-a228-655344585951 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.329364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0af87-9119-4c16-a228-655344585951 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.329632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0af87-9119-4c16-a228-655344585951 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.330352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0af87-9119-4c16-a228-655344585951 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:45.330654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0af87-9119-4c16-a228-655344585951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:45.332132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0af87-9119-4c16-a228-655344585951 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:45.355309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 400/803] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:45.355309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6bc6d-019d-416d-b42f-537957f33aa0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.355309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6bc6d-019d-416d-b42f-537957f33aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.355309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6bc6d-019d-416d-b42f-537957f33aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.370743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1ca6bc6d-019d-416d-b42f-537957f33aa0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:45.371916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1ca6bc6d-019d-416d-b42f-537957f33aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:45.439234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981c452-0542-41b8-95a3-9b29bff0fc34 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.440878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 401/804] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:45.442786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.443073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.443349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.461273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:45.462578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:45.469546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.476949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.477657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.477995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.478215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.478897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:45.499059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:45.499059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:45.499059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 404/805] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:45.499059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.544888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 402/806] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:45.544888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.573051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.628352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672aecb-2ea8-4986-800e-7ed0567ef3f0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.655485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623062d-c86a-4fb6-aa85-72bafd5dc7d0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.672947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.673981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.674372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.674766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.697942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.698571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.698955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.699229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.710826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:45.715605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:45.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:45.717480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:45.721068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 403/807] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:45.722128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.746831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.759210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:45.760707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:45.826620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.834437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90099982-facd-4836-8337-f5bf7b12a53b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.869289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:45.869942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:45.870040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 405/808] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:15:45.870257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:45.870498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/1e3ba7be5b48443697ad07dc4ffb04fe/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:45.870857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:45.872427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=1e3ba7be5b48443697ad07dc4ffb04fe, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:45.873911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:45.962681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:45.972613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae3579-5e6f-493d-a819-9c49e7f1c84b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:45.996399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 404/809] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:45 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/1e3ba7be5b48443697ad07dc4ffb04fe/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:45.997578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.008161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.008809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.009196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.009463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.010045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.011560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:46.022537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:46.023698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.024608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 406/810] 10.222.0.42 () {56 vars in 1129 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:46.025469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.145767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac23e-2979-4c7f-b7d6-d8f45305887b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.170444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532384bf-e18b-4ec2-ace8-437b8efb7d0f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.202738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.203362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.203647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.203919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.204589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.205318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.208340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.225247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 405/811] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.226413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.229385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.230297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.230771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.231344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] PATH_INFO: `/v3/services/network` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.232223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.234162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:46.245077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:46.246388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:15:46.247381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.248322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 407/812] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:46.249136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.261948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.270025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.353784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fb2-65fd-4d3c-9c5f-f32d0e460ad7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.354167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95000440-6b65-443b-9fbf-32cb84911f14 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.390345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.390798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.391127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.391410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.391611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.391705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.391947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.392040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.392575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.392893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.394311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.403792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 408/813] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.403792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.404018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.404419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:46.421004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:46.422004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:46.422907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:46.422907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:46.422907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:46.427547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 406/814] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:46.427932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.494518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.497325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.518432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d37c5-cef3-4c8a-abef-2793f9d744ec None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.537903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca183d8-aeca-4a59-8caa-5b14ffe9d4c0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.556248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.557187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.557469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.557792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.568550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.570038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.577028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.578236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.578715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.579526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.580630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] RBAC: Adding query filter params (name=network) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.581145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.583239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:46.585650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:46.586601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:46.587521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:46.587521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:46.587521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:46.587521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 409/815] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:46.587521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.590692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 407/816] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.592064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.702688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578f42-1508-4bd0-822d-83de05ff596b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.702874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af020-b900-4b93-8b54-0237870e0f1f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.733043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.738711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.738901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.739323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.739425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.739597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.739694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.739855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.739948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.740499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.740596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.740887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.746392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.746392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.746559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.746645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.752195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 410/817] 10.222.0.42 () {56 vars in 1124 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.752195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.763373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 408/818] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.763373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.856790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47791d-f679-4ee3-86ec-3c663a5308f1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.875314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9264d6a-ef86-4803-a22b-22ba2307e1c9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:46.891763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.892367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.892643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.892936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.893567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.893864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.895673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.908736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 411/819] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.909372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.926351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:46.926963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:46.927234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:46.927504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:46.928133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] RBAC: Adding query filter params (type=network) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:46.928429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:46.929827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:46.935004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 409/820] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:46.935801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:46.981367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:46.996075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:47.023630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a87893-863c-48cf-995b-fc6da027429c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.041442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8492a66-4b54-497c-9a5e-4bb8413a5d8a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.060119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.060735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.061063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.061344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.062002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.063366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.078495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.079123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.079399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.079667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.086741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.088165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:47.099659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:47.100750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 410/821] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:47.101704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:47.161632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 412/822] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:15:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:47.162442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:47.241090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57eddc-1d55-405f-a8a7-c1a48a39c4e5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.262666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:47.291447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.292469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.293110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.293756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.297927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152b29e-9299-4296-bd84-6864f3f9cbb6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.303984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.305852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.315247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:47.320686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:47.321720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:47.322964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:47.325575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 411/823] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:15:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:47.326698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:47.341416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.342591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.343251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.344102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.345286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.347254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.423614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 413/824] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:15:47 2022] POST /identity/v3/endpoints => generated 337 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:47.424503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:47.453605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf1b4e-143b-47c0-8df7-fd376ee7be27 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.501926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.502560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.503088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.503896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.505173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:47.505756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.508008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.526699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 412/825] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:47.532630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:47.586986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151bd6b-e610-41b7-8318-6b4df4357ab8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.597337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:47.665500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.666128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.666400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.666669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.667403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:47.667703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.669154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.690108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 414/826] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Oct 4 10:15:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:47.696815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:47.716583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694e49c-7962-4e83-83b6-ec1ce2369c71 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.730046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:47.757513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.758126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.758420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.758709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.759655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=adc1b4c4a2de4d97bb8019c9efc05570) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:47.759972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.762552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.769420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 413/827] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:47 2022] GET /identity/v3/role_assignments?user.id=adc1b4c4a2de4d97bb8019c9efc05570&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:47.821084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e846d1-5b47-4c02-932f-369c781c7a6b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:47.861818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:47.863273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:47.864075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:47.864540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:47.876575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:47.879026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:47.896316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:47.897572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:47.898695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:47.898695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:47.898695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:47.900526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 415/828] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:47.906197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:48.026867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:48.037976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af983b22-b429-43e6-9d19-be10e4a887ac None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:48.078312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:48.078933 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.079213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.079488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.080136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:48.080440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:48.082710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:48.103958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 414/829] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:48.105392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:48.194798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:48.214257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4534f5-b6ab-48fa-841a-78dcb7fbd9f2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:48.253709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:48.254352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.254629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.254901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.262032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:48.263552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-e2a0a5e5-3be4-4db5-8a4b-13563dd53fed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:48.274969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:48.275948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:48.276846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 416/830] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:48.282644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:48.289574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28473c-adba-441f-9cf8-09200e63f46c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.289860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28473c-adba-441f-9cf8-09200e63f46c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.290133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28473c-adba-441f-9cf8-09200e63f46c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.291716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 417/831] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:48.296843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.297157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.297432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.316775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:48.318011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:48.390154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48294cf-0861-4651-a745-f71553b1509b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:48.401139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 418/832] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:48.426204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:48.426819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.427098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.427375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.428166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:48.428477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:48.430021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:48.448187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 415/833] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:48.449645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:48.524424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:48.555402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bc90a-dbad-4e2d-af52-74ac73690c8c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:48.590873 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:48.591527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.591876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.592152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/64622f7e541046d69f64a4393316abd7/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.607643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=64622f7e541046d69f64a4393316abd7, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:48.619462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:48.687184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 419/834] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:15:48 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/64622f7e541046d69f64a4393316abd7/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:15:48.711941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:48.729675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:48.829753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb0276-01d8-46de-b5b8-47b75389dcab None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:48.871624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:48.872310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:48.872684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:48.873018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:48.885718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:48.890745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:48.892014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:48.893178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 416/835] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:48.899643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.010025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df969f-4d3e-4658-bdba-9e61197927fd None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:49.032423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:49.046954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:49.047567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.047842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.048111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.048778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:49.049110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:49.051068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:49.059897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 420/836] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:49.067941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.143005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:49.174657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29e9f3-996f-43b0-b506-f5b1200594d0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:49.211718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:49.212354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.212633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.212930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.224886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:49.226351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:49.241253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:49.242255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:49.243176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:49.243176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:49.243176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:49.243176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 417/837] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:49.248184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.365287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d2504-1ffd-40f6-8196-06a5535c8b2c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:49.400548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:49.401105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.401386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.401658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.402296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:49.402596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:49.403967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb915f1-6640-4d20-9ec6-36fa94a77b12 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:49.418596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 421/838] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:49.424169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.531706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b887-1ecd-48b6-84eb-9ef66656f3ed None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:49.533848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abf9ee-5a51-46c4-8889-caefa4eaec55 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.534128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abf9ee-5a51-46c4-8889-caefa4eaec55 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.534400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abf9ee-5a51-46c4-8889-caefa4eaec55 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.536561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 422/839] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:49.541538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.541815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.542086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.562058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:49.562347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:49.563551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:49.575642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:49.577434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.577678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.578264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.589510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:49.592053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:49.605954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:49.609118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:49.610085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:15:49.611152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:49.613715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 418/840] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:15:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:49.619770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.642322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 423/841] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:49.736955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db25ce-95ff-4f48-a3bf-16d38eb90947 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:49.776735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:49.777739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.778214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.778680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.779593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:49.780128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:49.781775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:49.801269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 419/842] 10.222.0.42 () {56 vars in 1135 bytes} [Tue Oct 4 10:15:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:49.807031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.908055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:49.914973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc336e75-1085-4e69-b603-ee06432877e0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:49.959129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:49.959755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:49.960037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:49.960312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:49.966606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.009128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.039982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.041499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.042849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d6129-001e-4f04-9a33-182ba6803a5e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.086503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.087166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.087444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.087717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.088321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.090490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:50.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:50.104650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:50.106684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:50.109321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 420/843] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:50.110359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.180895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 424/844] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:15:50.184036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.233400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712117f-a550-4df5-bc77-ff209b946f4f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.241206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.266543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.271164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.271773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.272053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.272335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.272978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:50.273370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.284492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f327fac-685b-4308-93f8-05b9511949c6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.284492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 421/845] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:50.284492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ab1eb-b1c9-4008-9147-a5276c2336a3 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.284492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ab1eb-b1c9-4008-9147-a5276c2336a3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.284492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ab1eb-b1c9-4008-9147-a5276c2336a3 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.284492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 422/846] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:50.285570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ab1eb-b1c9-4008-9147-a5276c2336a3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.307971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e27fa3-0295-45e2-a629-b534fd3f6ec9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.346274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.347241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.347519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.347794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.359061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.360493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f234af4d-7453-43e3-ab5a-f65ec75aaf3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:50.376277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:50.377488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 425/847] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.383465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.400137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:50.401232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ab1eb-b1c9-4008-9147-a5276c2336a3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.401344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:50.437249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.437972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.438369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.438738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.439457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:50.439862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.441942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.457429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 423/848] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:50.457429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.476375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 426/849] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:50.499739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.575632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851849-019b-40bf-8cb3-e0e9c0c0db20 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.614190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.614857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.615206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.615481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.616130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:50.616428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.617985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.632354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 424/850] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:50.638027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.698075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.744702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0472de4c-970b-45d2-b8e6-a7396e5072e9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.765070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.779661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.780337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.780570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.780843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.788565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.790331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:50.805703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:50.806680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:50.807679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:50.814566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 427/851] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:50.815780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.873381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:50.881740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306fe32-fc9b-4d15-95dc-d6ba6d41bc8c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.925740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:50.927342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:50.927854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:50.928326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:50.929132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:50.931200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:50.943688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:50.944686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:50.945612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 425/852] 10.222.0.42 () {56 vars in 1132 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:50.952348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:50.956190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d080-64c7-4f66-9ea4-ca6ee6ce87c1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:50.975816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:51.005097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:51.005731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.006013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.006289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.006993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:51.007297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:51.008767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:51.028962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 428/853] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:51.034416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:51.088756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e84a9d-c453-41fd-bf55-38cd5fa434c8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:51.129862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:51.130555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.130934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.131216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.131840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:51.132140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:51.133540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7e5ad-179f-4bca-a8ef-4c89ae7e95a3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:51.138261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 426/854] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:15:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:51.139296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c829-838f-4532-a38b-0a028568dadb None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.139572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c829-838f-4532-a38b-0a028568dadb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.139846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c829-838f-4532-a38b-0a028568dadb None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.144892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 427/855] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:51.144892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0c829-838f-4532-a38b-0a028568dadb None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:51.154910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba09440-8985-4644-bf71-70302f0fcae3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:51.179592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:51.191520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:51.192811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.192811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.192811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.193381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=1e3ba7be5b48443697ad07dc4ffb04fe) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:51.193683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:51.195099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec005ec6-f235-42ad-8909-df16d77d01d8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:51.201619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 429/856] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:51 2022] GET /identity/v3/role_assignments?user.id=1e3ba7be5b48443697ad07dc4ffb04fe&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:51.202905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.203190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.204541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.225983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:51.227310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:51.265540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:51.304516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0c829-838f-4532-a38b-0a028568dadb None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:51.317637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 430/857] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:51.349339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:51.349964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.350243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.350518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.351158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:51.351525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:51.353099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:51.358815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 428/858] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:15:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:51.597629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:51.709337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c6a26-38f9-40fa-8401-d6fabd1f1dbf None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:51.747411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:51.748295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.748721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.749196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.760803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:51.765916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:51.767064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:15:51.768137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:51.775994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 431/859] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:51.777678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:51.866650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:51.885091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ae9c-53df-4b9b-be0d-d2c2e6523937 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:51.921394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:51.922001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:51.922277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:51.922544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:51.923465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:51.923770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:51.925241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:51.934036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 429/860] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:51.939781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:51.972580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:52.049395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30db4e-819d-4bfc-8248-c374a85363e2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:52.086110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:52.087095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.087603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.088089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.098971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:52.100686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:52.116121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:52.117180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:52.118106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:52.118106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:52.118106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:52.124453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 432/861] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:52.126050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:52.237517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0cf54-3782-41f1-8d6c-14a0c1a5113f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:52.275913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:52.276896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.277405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.277895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.278814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:52.279336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:52.280978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:52.300634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 430/862] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:52.302582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:52.388072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:52.407631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c854641-a42a-4cda-a3d3-9b01e2519fb8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:52.443056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:52.443686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.443965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.444236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.452302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:52.453769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:52.465886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:52.466830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:52.467788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:52.471539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 433/863] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:15:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:52.474007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:52.485076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:52.582065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c483-16cc-428b-b49a-b254e88a95d3 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:52.619441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:52.620264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.620721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.621178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.622058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:52.622521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:52.628884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbdb01-140a-48be-9f15-c5d6e0f7d6af None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:52.648424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 431/864] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Oct 4 10:15:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:52.649855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:52.770523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a11f9-4eb9-4c04-aae4-b71b3ce878ee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:52.805148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:52.805757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.806034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.806304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.818057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:52.819590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af59a2-5131-4175-a97e-db4677428479 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.820072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af59a2-5131-4175-a97e-db4677428479 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.820254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af59a2-5131-4175-a97e-db4677428479 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.822372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 432/865] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:52.827975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:52.828464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:52.828925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:52.848062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:52.849611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:52.888679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:52.890278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:52.923159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:52.936898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 433/866] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:53.010063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 434/867] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:15:53.015835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:53.121553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e30662-6408-4a1e-8224-f04e661820b9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:53.158451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:53.159096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.159496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.159897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.170569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:53.172028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:53.187915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:53.188915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:53.190236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:53.190236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:53.190236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:53.190236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 434/868] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:53.197023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:53.234825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:53.242544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:53.288208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:53.311813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fadad4-0df7-4479-8a05-0b79a6e741d9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:53.346510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d14cc0-943f-4125-864d-316afc9c16a0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:53.352853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:53.353491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.353770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.354042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.354681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:53.355413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:53.356802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:53.371557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 435/869] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:53.376780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:53.389407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a04ca-7a4d-4371-a775-5c094eff1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:53.390352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a04ca-7a4d-4371-a775-5c094eff1f11 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.390961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a04ca-7a4d-4371-a775-5c094eff1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.391296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a04ca-7a4d-4371-a775-5c094eff1f11 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.392084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a04ca-7a4d-4371-a775-5c094eff1f11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:53.393628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a04ca-7a4d-4371-a775-5c094eff1f11 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:53.468119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 435/870] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:15:53 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:15:53.506611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:53.510321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e132-06aa-469e-b065-0265e4a62eee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:53.560842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:53.561517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.561794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.562065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.570209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:53.571765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:53.585151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:53.586114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:53.587444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 436/871] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:53.591481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50921e94-0cfa-4d52-a999-d2c2ce71ba6a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.591766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50921e94-0cfa-4d52-a999-d2c2ce71ba6a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.592051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50921e94-0cfa-4d52-a999-d2c2ce71ba6a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.593719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 436/872] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:53.595272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:53.600423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67c063-2a9a-43d1-85e2-2271f3dca98e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.600593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67c063-2a9a-43d1-85e2-2271f3dca98e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.600885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67c063-2a9a-43d1-85e2-2271f3dca98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.621874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2d67c063-2a9a-43d1-85e2-2271f3dca98e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:53.623078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2d67c063-2a9a-43d1-85e2-2271f3dca98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:53.699980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 437/873] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:53.701114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75126bce-4659-4689-a6ec-38aa68a3fc99 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.701381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75126bce-4659-4689-a6ec-38aa68a3fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.701663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75126bce-4659-4689-a6ec-38aa68a3fc99 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.703116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 438/874] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:53.708487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.708765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.709110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.727031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc436fb-5290-4bc3-a1c0-443587effec6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:53.729475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:53.731552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:53.765533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:53.766538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.767226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.768008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.769398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:53.770075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:53.771907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:53.786034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 437/875] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:53.791769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:53.793234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:53.814383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:53.821870 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 439/876] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:53.933494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c3f32-28d9-49fb-bf77-1398499283d3 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:53.969616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:53.970224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:53.970501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:53.970777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:53.971495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=64622f7e541046d69f64a4393316abd7) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:53.971800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:53.974063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:53.985059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 438/877] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:53 2022] GET /identity/v3/role_assignments?user.id=64622f7e541046d69f64a4393316abd7&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:54.000182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:54.091712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:54.123883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab6c87-a377-4441-a48c-0e2da99af557 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:54.165612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:54.166338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:54.166612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:54.166897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:54.167512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:54.168896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:54.180579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:54.181639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:54.182491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 440/878] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:15:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:54.189143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:54.198426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:54.220335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b2f0d-989b-4a58-8855-6a133d972adc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:54.232100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:54.268514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f9f2a-9703-4523-a9a4-130ea04d2908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:54.269385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f9f2a-9703-4523-a9a4-130ea04d2908 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:54.269862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f9f2a-9703-4523-a9a4-130ea04d2908 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:54.270358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f9f2a-9703-4523-a9a4-130ea04d2908 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:54.271481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f9f2a-9703-4523-a9a4-130ea04d2908 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:54.273443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f9f2a-9703-4523-a9a4-130ea04d2908 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:54.318558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311f92f-8374-4ae4-bdb2-8fdbef55ff12 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:54.357692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:54.358763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:54.359243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:54.359691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:54.360569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:54.361068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:54.363234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:54.368496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 441/879] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:15:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:54.374256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:54.385997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 439/880] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:15:54 2022] POST /identity/v3/services => generated 269 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:15:54.491520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:54.498738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b672c-5400-4477-bae1-a9bd989446ad None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:54.542374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:54.542900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:54.543177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:54.543452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:54.544079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:54.544380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:54.545921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb77f8d-101b-4364-8572-ec15bdf3668b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:54.551488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 442/881] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:15:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:55.816490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe477dd0-63eb-4379-98ed-3f52753f4d22 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:55.817069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe477dd0-63eb-4379-98ed-3f52753f4d22 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:55.817069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe477dd0-63eb-4379-98ed-3f52753f4d22 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:55.818691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 440/882] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:55.824616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb564c96-e2f3-4091-9eb9-eb971b522ba6 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:55.825061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb564c96-e2f3-4091-9eb9-eb971b522ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:55.825253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb564c96-e2f3-4091-9eb9-eb971b522ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:55.843812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-eb564c96-e2f3-4091-9eb9-eb971b522ba6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:55.845385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-eb564c96-e2f3-4091-9eb9-eb971b522ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:55.916615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 443/883] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:56.183731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe477dd0-63eb-4379-98ed-3f52753f4d22 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:56.298842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe477dd0-63eb-4379-98ed-3f52753f4d22 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:56.324369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f021d-07cd-44a5-a35c-1504441b7fc7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.324717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f021d-07cd-44a5-a35c-1504441b7fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.324951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f021d-07cd-44a5-a35c-1504441b7fc7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.326515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 444/884] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:56.332152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.332434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.332705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.337758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:56.338392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.338670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.338942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.339524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:56.340994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:56.350710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:56.351761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:56.352619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:56.354857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:56.356254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:56.356902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 441/885] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:56.358099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:56.363022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:56.445203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 445/886] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:56.464906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31701e5-02b0-4439-a6f4-76047dc9aa56 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:56.500447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:56.501134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.501416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.501686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.502308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:56.502608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:56.504534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:56.514050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 442/887] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:56.515392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:56.535136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:56.625554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608eaa6-cf04-4eb7-9e1d-6a0c4aa93478 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:56.662784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:56.663399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.663680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.663953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.664587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:56.664908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:56.666288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:56.673398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 446/888] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:56.679213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:56.730608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:56.755800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:56.790182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de4268-0507-4ce9-92fd-7d63ee120780 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:56.822681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:56.826338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:56.826953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.827229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.827583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.828200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] RBAC: Adding query filter params (interface=public, service_id=da3850c7a9fb43f68ccde799fc17c6d4, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:56.828507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:56.830141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2040f73-58c8-4516-8e68-b78937628f6c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:56.838081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 443/889] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/endpoints?service_id=da3850c7a9fb43f68ccde799fc17c6d4&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:56.839345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666526f-2089-4c21-b565-e4eca62d00ec None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.839650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666526f-2089-4c21-b565-e4eca62d00ec None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.839908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666526f-2089-4c21-b565-e4eca62d00ec None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.841315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 444/890] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:56.842233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a8a20-5098-4f57-95a9-e9d4ae3c2387 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.842752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a8a20-5098-4f57-95a9-e9d4ae3c2387 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.843038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a8a20-5098-4f57-95a9-e9d4ae3c2387 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.844207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 445/891] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:56.845020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a8a20-5098-4f57-95a9-e9d4ae3c2387 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:56.849194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee32429-b789-4021-9338-41b52145a824 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:56.883563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fb9f1-58e3-4ef6-aae7-e6f98954aac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:56.884186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb9f1-58e3-4ef6-aae7-e6f98954aac5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.884465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb9f1-58e3-4ef6-aae7-e6f98954aac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.884739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb9f1-58e3-4ef6-aae7-e6f98954aac5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.894672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fb9f1-58e3-4ef6-aae7-e6f98954aac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-184fb9f1-58e3-4ef6-aae7-e6f98954aac5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:15:56.899808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:56.900740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:56.901624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 447/892] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:56.903131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1995e9e-bab2-40f3-a39d-be09de6fba03 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.903131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1995e9e-bab2-40f3-a39d-be09de6fba03 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.903131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1995e9e-bab2-40f3-a39d-be09de6fba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.922125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e1995e9e-bab2-40f3-a39d-be09de6fba03 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:56.923347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e1995e9e-bab2-40f3-a39d-be09de6fba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:56.950505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a8a20-5098-4f57-95a9-e9d4ae3c2387 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:56.978949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:56.984674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:56.985713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.986214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.986864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:56.987228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:56.990308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:56.993016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 448/893] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:56.994811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:56.995171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:56.995423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.003877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 446/894] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:15:57.003877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.030878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:57.034501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:57.124825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:57.130977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 449/895] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:15:57.132345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd86e4-7ec2-41db-b2d5-bbff2f633190 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.172473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.173212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.173493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.173768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.174441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:57.174739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.177455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.186232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 447/896] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:15:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:57.191874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.277801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.306691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1912afa-02ec-4cec-8ba5-9d635e1f7ad2 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.349169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:57.353773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.354464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.365897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.365897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.367680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.369173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:57.386250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:57.387253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:57.388157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:57.388157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:57.388157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:57.388157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 450/897] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:57.394410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.409016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf027a-efce-49ca-a8a3-8f04638ee124 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.445989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.446617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.446896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.447426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.448010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.449405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.454489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:57.458629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:57.459570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:57.460464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 448/898] 10.222.0.42 () {56 vars in 1132 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:15:57.461311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.513472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca747374-f5f7-4856-9581-4d649966da6e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.549326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.550951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.551453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.552038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.553013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:57.553526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.555125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.567518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea5df7-d26c-43ff-94b7-f8c74c3018ee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.573059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 451/899] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:57.573948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.592006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:57.604309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.605221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.605680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.605950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.606594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.607949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.670034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 449/900] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:15:57 2022] POST /identity/v3/services => generated 241 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:15:57.670840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.712226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1475-d931-4194-87cc-fb71011396f7 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.740132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:57.773527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.774148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.774430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.774705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.775356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:57.775656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.777430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.788227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 452/901] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:57.793809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.821785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19a454-53f8-48b3-a30b-d42c2c8512e0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.860768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.861591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.861807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.862971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.872433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.873920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:15:57.887431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:57.888391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:15:57.889427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:15:57.891265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 450/902] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:57.898240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:57.941918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074d333-2aef-4e0f-88a5-bdcd96db12f0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:57.952225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:57.992391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:57.993036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:57.993361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:57.993953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:57.994865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:57.995170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:57.996713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.004364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 453/903] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:58.010149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:58.021051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e36e0-289c-4f1d-b636-14775976ec0f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:58.051077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:58.064010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:58.064831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:58.065287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:58.066236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:58.067145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:58.067602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:58.069339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.094712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 451/904] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Oct 4 10:15:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:58.095897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:58.128566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70104574-9c8e-41a5-a5ad-71901496095f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:58.165618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:58.166547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:58.166977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:58.167398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:58.168261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] RBAC: Adding query filter params (interface=public, service_id=0ed93d62a6e7445186a4552869deeda3, region_id=RegionOne) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:58.168720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:58.170279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.180583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 454/905] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:15:58 2022] GET /identity/v3/endpoints?service_id=0ed93d62a6e7445186a4552869deeda3&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:58.181714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:58.226695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:58.231323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d2861-1198-4807-849f-78d10615b4a6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:58.275838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:58.276703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:58.277150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:58.277608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:58.303080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:58.306706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257c5a9-6e92-4553-b20c-4fbc2a4726c9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:58.354442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:58.355218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:58.355503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:58.355782 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:58.356807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:58.358487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.372886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 455/906] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:15:58 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:15:58.373115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:58.373599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.486022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:58.490558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 452/907] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:15:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:15:58.498128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:58.626379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09370bac-894b-4257-a053-8e7ad001b1f2 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:58.665890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:58.666533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:58.666814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:58.667087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:58.679905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:58.681399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:15:58.696540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:58.697852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:15:58.698749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:15:58.698749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:15:58.698749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:58.698749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 456/908] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:58.705327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:58.809991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b272a2-7029-44be-8301-a8814862b51c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:58.850357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:58.850969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:58.851248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:58.851518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:58.852161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:58.852462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:58.853859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:58.866124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 453/909] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:58.872076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:58.892193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:58.985714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c75dfc-48c8-410a-9daf-aa6324ee8254 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:59.027509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:59.028145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.028586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.028879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.036509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:59.038061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-494f564a-fb24-4be6-90ea-fc382c48e1e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:15:59.050728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:59.051694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:59.052723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 457/910] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:59.059696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:59.127184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:59.175783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1daa43-cde7-4f1b-8088-eefd15d35d1e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:59.211299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:59.211913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.212192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.212465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.213182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:59.213485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:59.214909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:59.222514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d344c0-e72f-4d39-be16-35257e0fa781 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.222794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d344c0-e72f-4d39-be16-35257e0fa781 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.223066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d344c0-e72f-4d39-be16-35257e0fa781 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.224667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 458/911] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:15:59.228190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 454/912] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:59.229887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.230169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.230443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.235152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:59.250007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:59.251698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:59.324220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 459/913] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:15:59.348093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d38c0e7-3123-40eb-bde9-9666d2e62b21 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:59.370262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:59.388067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:59.388067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.388067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.388067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.388474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] RBAC: Adding query filter params (role.id=1ca160d1b03847e8a6058aa73a00705d, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=64622f7e541046d69f64a4393316abd7) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:59.388474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:59.389215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:59.396310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 455/914] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:15:59 2022] GET /identity/v3/role_assignments?user.id=64622f7e541046d69f64a4393316abd7&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=1ca160d1b03847e8a6058aa73a00705d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:15:59.603532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:59.612136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:59.710774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e044728-3366-4770-8e04-66c200acfb79 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:59.747128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:59.747810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.748087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.748354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.748964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:59.750323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:15:59.759985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:15:59.761107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:15:59.761968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 460/915] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:15:59.767041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:59.872259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdce11-c574-4bf8-a867-c23b869daf30 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:59.907468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:59.908075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:59.908351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:59.908620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:59.909824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:15:59.910134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:59.911485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:59.916412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 456/916] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:15:59.925985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:00.000676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:00.032519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f9ed8-62d4-449f-9a22-2b09edd52c27 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:00.074109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:00.074726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.075004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.075276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.075910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:00.076218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:00.077623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e62be-ccfe-44d5-9818-1c67691be683 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:00.083871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 461/917] 10.222.0.42 () {56 vars in 1114 bytes} [Tue Oct 4 10:15:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:00.089175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:00.094214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b70c-0ae3-4156-97a1-67d617e1d34e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.094496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b70c-0ae3-4156-97a1-67d617e1d34e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.094773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096b70c-0ae3-4156-97a1-67d617e1d34e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.096410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 462/918] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:00.102544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.102833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.103144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.123015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:00.124206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:00.141628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:00.201782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef65d-7fea-4aaf-91d8-578c155de862 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:00.205591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 463/919] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:00.250527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70619b-c155-4722-a782-88691f73f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:00.251201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70619b-c155-4722-a782-88691f73f810 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.251418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70619b-c155-4722-a782-88691f73f810 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.251689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70619b-c155-4722-a782-88691f73f810 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.252353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70619b-c155-4722-a782-88691f73f810 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:00.254647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70619b-c155-4722-a782-88691f73f810 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:00.345157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 457/920] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:16:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:00.514852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:00.582681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:00.637496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a109a-55a9-4e87-9d87-2890d7275f8c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:00.678496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:00.679209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.679489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.679761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.680343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:00.681746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:00.690887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:00.691953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:00.692810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:00.696118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 464/921] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:16:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:00.698148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff87b6d-bb6d-428b-b4df-9c1e5403083d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.698731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff87b6d-bb6d-428b-b4df-9c1e5403083d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.699081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff87b6d-bb6d-428b-b4df-9c1e5403083d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.699374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:00.700830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 458/922] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:00.707403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.707751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.708035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.726797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:00.728077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:00.805754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 459/923] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:00.819034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20a235-f9d1-44cf-acfe-b79b2366046b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:00.845271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:00.854561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:00.855165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:00.855473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:00.855758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:00.856389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:00.856690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:00.858073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:00.863056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 465/924] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:00.868713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:00.904057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:00.973025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babb012-b10e-4d71-82c6-5eb863d4c647 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.007602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.008222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.008505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.008783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.009444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:01.009749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.011126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.016228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 460/925] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:01.021696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:01.115438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:01.137874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f31091-77f2-482d-80df-c3d0a37d560e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.174717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.176068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.176349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.176622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.177716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] RBAC: Adding query filter params (interface=public, service_id=3d08728c4b12420193f92572981df859, region_id=RegionOne) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:01.177829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.179161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.185328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 466/926] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:16:01 2022] GET /identity/v3/endpoints?service_id=3d08728c4b12420193f92572981df859&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:01.191076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:01.221484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:01.231057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75869621-d5f9-43a4-bf88-47b45ecf63e2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.251128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:01.282102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.282735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.283161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.283379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.283901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.285861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:01.296830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:01.297860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:01.304897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 461/927] 10.222.0.42 () {56 vars in 1132 bytes} [Tue Oct 4 10:16:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:01.307296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:01.313839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdb717-f458-4550-b129-d30d6261fe07 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.356074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.357194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.357743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.358225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.359224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.360917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.369523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 467/928] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:16:01 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:16:01.437595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee5613-692f-443d-9830-3450e3e46023 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.480944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.481957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.482467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.482939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.483789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:01.484299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.486040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.492631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 462/929] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:16:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:01.497962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:01.620982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3ba0a-562a-4fa9-a322-5c592171321e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.639725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:01.663157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.663770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.664100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.664391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.665069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:01.665424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.666813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3733dd-0631-4fa0-8712-9a08c37da484 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.672158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 468/930] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:16:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:01.677441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:01.707128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:01.725691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526c82-fe2e-4932-b658-e40d487237b7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.728296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526c82-fe2e-4932-b658-e40d487237b7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.728296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526c82-fe2e-4932-b658-e40d487237b7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.728296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 469/931] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:01.733333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.733623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.733906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.753691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:01.755055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:01.795606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13c29b-f84b-4cc2-b36c-f2ab8cd0f757 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:01.839108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 470/932] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:01.842096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:01.842994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:01.843470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:01.843951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:01.844832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:01.846425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:01.957624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 463/933] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:16:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:02.114224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:02.222028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5e36-c1eb-4945-beb3-6563c8939773 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:02.233519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:02.264590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:02.265250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.265525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.265793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.277169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:02.281866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:02.282793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:02.283642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 471/934] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Oct 4 10:16:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:02.289109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:02.376586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:02.395863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a310bce-c33f-417e-b6c9-a9f6e55f43c9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:02.433837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:02.434782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.435237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.435681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.436513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:02.437026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:02.438556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:02.448384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 464/935] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:16:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:02.454521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:02.560444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251c92ea-71cd-44fe-905b-75318cc89a17 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:02.599449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:02.600114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.600389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.600662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.615490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:02.615490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-c955f89f-81c2-47f6-8ce3-811ef3216455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:02.630646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:02.631591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:02.632491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:02.632491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:02.632491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:02.632491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 472/936] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:02.637618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:02.710315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9884cb-db7d-4a7f-8177-d88ad7b0f8f8 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.710932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9884cb-db7d-4a7f-8177-d88ad7b0f8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.711401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9884cb-db7d-4a7f-8177-d88ad7b0f8f8 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.713201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 473/937] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:02.718249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.718544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.718823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.740425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:02.741102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08250f1-55a5-4b33-8c42-0bef555feee8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:02.741685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:02.761162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:02.785827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:02.786850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.787325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.787862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.788923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:02.790290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:02.791908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:02.807044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 465/938] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:02.813032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:02.817290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:02.840145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 474/939] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:02.923587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31d4d3-a4e0-4f3a-8a60-ff8df6a57322 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:02.965634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:02.966270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:02.966515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:02.966785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:02.975239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:02.976653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:16:02.987678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:02.988685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:02.989727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 466/940] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:16:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:02.994703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.094858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a001a-5166-4a19-9dcc-eeddc92823fd None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.129838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.130493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.130716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.130963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.131681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:03.131979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.133427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.142218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.155385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 475/941] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Oct 4 10:16:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:03.163576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.174672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:03.222626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:03.265870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6eb94-b9b5-487b-8ce6-6988045200e2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.303469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.304084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.304363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.304636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.304953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c79065-fd30-408f-a1e4-589fd84ac543 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.305345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.306910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.309521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING keystone.server.flask.common [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 10:16:03.345468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.346182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.346844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.346965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.358909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.359443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.sql.core [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 10:16:03.359443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 10:16:03.359443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [parameters: {'id': '6c016eb246b54ee6a68e75f33a27b79b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 10:16:03.359443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81561) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 10:16:03.360564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:03.375934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:03.377017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:03.383418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:03.383418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:03.383418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:03.383418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 476/942] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:03.385105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6c016eb246b54ee6a68e75f33a27b79b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6c016eb246b54ee6a68e75f33a27b79b. Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:16:03.385855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 10:16:03.386724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application yield resource Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise self.value Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application session.commit() Oct 04 10:16:03.387693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.flush() Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application compat.raise_( Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 10:16:03.388607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 10:16:03.389592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:16:03.390410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [parameters: {'id': '6c016eb246b54ee6a68e75f33a27b79b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.391298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:03.392101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6c016eb246b54ee6a68e75f33a27b79b. Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.393358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 467/943] 10.222.0.42 () {60 vars in 1140 bytes} [Tue Oct 4 10:16:03 2022] POST /identity/v3/projects => generated 262 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:03.397383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.463846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:03.517423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb51a8c-9c25-4b05-a296-a81513fe3eb8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.527914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4855b8c5-06dd-46ee-b0f3-88f9c16c809b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.551784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:03.565984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.566525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.566805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.567243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.567891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:03.568197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.568563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.569445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.569625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.570105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.570645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.578751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.580814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.584937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 477/944] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:03.594392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:16:03.601076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:03.602064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:03.602998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:03.605895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 468/945] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:03.613666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.713073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d20aa6-d74d-4fe9-a40b-e636727d51f2 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.731767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:03.750061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.752117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.752117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.752117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.754477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f149a-7a33-43c1-bd4b-68b1f1371935 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.760137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.762046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-a09603ca-1673-4550-bbac-122e8170c259 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:16:03.775371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:03.776595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:03.777634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:03.781170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 478/946] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:03.782644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774215a-4f85-4114-a4b5-494091798958 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.782936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774215a-4f85-4114-a4b5-494091798958 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.783218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774215a-4f85-4114-a4b5-494091798958 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.785279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 479/947] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:03.785842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774215a-4f85-4114-a4b5-494091798958 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:03.799863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.800497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.800747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.802743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.802743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:03.802743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.803839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6659639-567b-4cae-9250-34faa9d08599 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.836264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 469/948] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:03.836264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.836264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.836264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.857931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:03.860513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:03.912061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774215a-4f85-4114-a4b5-494091798958 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:03.913804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:03.958511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 470/949] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:03.958721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:03.959312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:03.959631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:03.959870 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:03.960575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:03.968917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:03.968917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:03.976701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 480/950] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:03.984321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.094339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c278809-0571-4bb3-ada1-8ee06a0e46e6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:04.131502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a525773-6d4d-4dd6-b99a-8c9700b46aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:04.132107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525773-6d4d-4dd6-b99a-8c9700b46aa6 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.132385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525773-6d4d-4dd6-b99a-8c9700b46aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.132664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a525773-6d4d-4dd6-b99a-8c9700b46aa6 None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/64622f7e541046d69f64a4393316abd7/roles/1ca160d1b03847e8a6058aa73a00705d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.134892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a525773-6d4d-4dd6-b99a-8c9700b46aa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=64622f7e541046d69f64a4393316abd7, role_id=1ca160d1b03847e8a6058aa73a00705d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:04.136409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a525773-6d4d-4dd6-b99a-8c9700b46aa6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:04.205529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 471/951] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:16:03 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/64622f7e541046d69f64a4393316abd7/roles/1ca160d1b03847e8a6058aa73a00705d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:16:04.233141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.285231 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:04.287434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2c40e-11a1-4a50-b4fd-ac3bd081d914 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.287717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2c40e-11a1-4a50-b4fd-ac3bd081d914 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.287992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2c40e-11a1-4a50-b4fd-ac3bd081d914 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.289671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 472/952] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:04.295367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.295641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.295912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.318361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:04.319602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:04.353245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:04.362042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4b3b7-02f0-4598-bbfb-4fbb15705184 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:04.406221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 473/953] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:04.410810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:04.411464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.411788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.412062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.412660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:04.414162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:04.426824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:04.428098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:04.429029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 481/954] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:16:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:04.435755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.539056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9f1db-05dd-4c7a-be54-16084acbfbc3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:04.575109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:04.575732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.576014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.576287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.576956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:04.577263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:04.578634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:04.584425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 474/955] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:04.590043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.673542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.701283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:04.706666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b697f-cc5e-4873-af4a-9aea2002b8aa None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:04.708698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:04.751291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:04.752002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.752282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.752560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.753843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:04.754145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:04.755514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:04.764623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 482/956] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:04.768009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.791783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68748692-ffc5-40c9-b0aa-61f135292a10 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:04.829908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:04.831037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.832008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.832506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.833353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:04.835024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:04.844623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:04.845846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:04.847914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:04.849847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 475/957] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:04.855341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:04.894596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd07d4-f89e-4dfe-9c24-055ceb743230 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:04.933243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:04.933942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:04.934405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:04.934938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:04.935553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:04.937067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:04.969525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:04.982665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30ebbb-c749-408e-8c3f-612673b87881 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:05.016620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:05.017409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:05.017639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:05.017944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:05.019242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:05.019552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:05.027788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f959e57-b0f3-4ff9-9536-66186d07434e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:05.027985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 483/958] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:16:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:16:05.033258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 476/959] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:05.043915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:05.073668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:05.166200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce06d3-671e-4d00-b8df-e068abefbd8f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:05.211588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:05.212577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:05.213179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:05.213680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:05.214553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:05.215059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:05.216798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101b23b-7faa-4938-a691-180f5eee223b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:05.223272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 484/960] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:06.216892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1b9d-a91a-4f49-866f-e9b8796ac6c2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:06.217471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1b9d-a91a-4f49-866f-e9b8796ac6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:06.217471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f1b9d-a91a-4f49-866f-e9b8796ac6c2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:06.219062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 477/961] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:06.224657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21406-a1e1-4b6f-bf0e-dacbb9c3930c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:06.225087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21406-a1e1-4b6f-bf0e-dacbb9c3930c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:06.225273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21406-a1e1-4b6f-bf0e-dacbb9c3930c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:06.244212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e1e21406-a1e1-4b6f-bf0e-dacbb9c3930c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:06.245449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e1e21406-a1e1-4b6f-bf0e-dacbb9c3930c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:06.328978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 485/962] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:06.532135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57c8fc-0b7a-4852-ad0c-30c4f19ae02f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:06.532519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57c8fc-0b7a-4852-ad0c-30c4f19ae02f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:06.532707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57c8fc-0b7a-4852-ad0c-30c4f19ae02f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:06.534336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 478/963] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:06.539417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:06.539708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:06.539979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:06.558739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:06.559910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:06.568030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:06.606927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57c8fc-0b7a-4852-ad0c-30c4f19ae02f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:06.630760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 486/964] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:06.726823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57c8fc-0b7a-4852-ad0c-30c4f19ae02f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:06.763248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:06.763859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:06.764140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:06.764474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:06.765157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:06.766692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:06.817259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.sql.core [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 10:16:06.817259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 10:16:06.817259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [parameters: {'user_id': '92519b5608524fe0b1093dd40a52d78d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 10:16:06.817259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81561) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Oct 04 10:16:06.825380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:16:06.826241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application yield resource Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise self.value Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application session.commit() Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 10:16:06.827199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.session.flush() Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application compat.raise_( Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 10:16:06.828201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:16:06.837410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application result.read() Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:16:06.839075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [parameters: {'user_id': '92519b5608524fe0b1093dd40a52d78d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:06.840793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 10:16:06.842529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:06.844453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 479/965] 10.222.0.42 () {60 vars in 1131 bytes} [Tue Oct 4 10:16:06 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:06.845092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:06.925291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:06.943587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:06.954079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e786f-69cf-4534-a267-91a62e7384d5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:06.991220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:06.991829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:06.992262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:06.992632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.000479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.002365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:16:07.014769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:07.016027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:16:07.017308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:07.023489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 487/966] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:07.025757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.030081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:07.053435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373014fc-cf95-4550-9a3e-904e8c4d9835 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.096072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be5108-4615-4258-ba11-d563db143648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.097510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be5108-4615-4258-ba11-d563db143648 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.097994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be5108-4615-4258-ba11-d563db143648 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.098451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be5108-4615-4258-ba11-d563db143648 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.110084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be5108-4615-4258-ba11-d563db143648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-83be5108-4615-4258-ba11-d563db143648 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:07.116017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:07.117217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:07.118291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 480/967] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Oct 4 10:16:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:07.125907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be5108-4615-4258-ba11-d563db143648 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.146372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9a22-ff97-41e1-b827-d65eada98381 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.183049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.183907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.184223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.184603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.185622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:07.186049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.187776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9626d-da84-4e41-92c6-cd78863cd6c7 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.223416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:07.229663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 488/968] 10.222.0.42 () {56 vars in 1101 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:07.256057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be5108-4615-4258-ba11-d563db143648 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.297825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.298847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.299340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.300413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.301635 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:07.302225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.304183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.312065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176056-9980-44d9-be1f-87ba13e4ef82 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.312416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176056-9980-44d9-be1f-87ba13e4ef82 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.312616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176056-9980-44d9-be1f-87ba13e4ef82 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.314422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 489/969] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:07.319803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 481/970] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:07.320666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ccb7-b0e1-428d-b7a9-5756b5626a3a None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.325090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ccb7-b0e1-428d-b7a9-5756b5626a3a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.325090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ccb7-b0e1-428d-b7a9-5756b5626a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.326760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.343178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-77b2ccb7-b0e1-428d-b7a9-5756b5626a3a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:07.344517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-77b2ccb7-b0e1-428d-b7a9-5756b5626a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:07.432635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:07.446328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 490/971] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:07.448329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97afd-09e9-49ff-b486-bb7386e2016c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.448734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97afd-09e9-49ff-b486-bb7386e2016c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.449069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f97afd-09e9-49ff-b486-bb7386e2016c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.450523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 491/972] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:07.451373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311441-c3a7-410a-8e80-3efe776d4676 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.463006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.463006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.463006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.478974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:07.480162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:07.494623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.495356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.495712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.495993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.502506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:07.512367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.515068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:07.533911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:07.535236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:07.536120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:07.536120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:07.536120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:07.537006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 482/973] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:07.541118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.566197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 492/974] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:07.663263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429907e-8234-4918-b47b-979b2217ff9c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.697992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.698528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.698802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.699070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.699698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:07.699997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.701383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.713504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 483/975] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:07.718872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.724280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.775298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:07.825186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479340d0-348e-46eb-8b04-a8ddad6abcd7 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.834660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933da6d8-07e3-4e0c-8666-87422d891822 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.848379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:07.861830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.862962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.863268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.863508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.869565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:07.870184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:07.870458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:07.870620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.870717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:07.871284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:07.872069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.872625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:07.882254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:07.883404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:07.884277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 484/976] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:07.885108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:16:07.885191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:07.886174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:16:07.887149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:07.888638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 493/977] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:07.889511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:07.989419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d6446-acaa-480f-925c-365db3acf1bf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:07.990757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb5729-4740-4bf5-bcca-af7ca576fffc None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.001615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:08.023308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.024198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.024545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.024817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.025040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.025460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.025741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.025870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.026017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.026648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:08.026953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.027481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.028307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.032885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 494/978] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:08.033758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.078359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:08.086425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 485/979] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:16:07 2022] POST /identity/v3/services => generated 242 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:08.088514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.138717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5d7c0-8572-4716-9211-ee268fce68f8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.187566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.188292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.188644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.189027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.189854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:08.191032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.192689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.217035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 495/980] 10.222.0.42 () {56 vars in 1149 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:08.218303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.223891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dedd1e-eac0-4748-b8de-097c0a96bdc2 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.263160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.263835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.264116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.264396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.266255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:08.266255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.266719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.273388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 486/981] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:08.281963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.326983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:08.329689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:08.333663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2392177-ad9c-420c-8d9b-8ab896bb746f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.369326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.369943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.370223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.370545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.405644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da736e45-9ac9-444f-92bb-4a07ea6411cf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.442145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.442389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.443034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.443320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.443596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.443773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.444303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] RBAC: Adding query filter params (interface=admin, service_id=3d08728c4b12420193f92572981df859, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:08.444618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.446198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.457778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 487/982] 10.222.0.42 () {56 vars in 1249 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/endpoints?service_id=3d08728c4b12420193f92572981df859&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:08.458635 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.547810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:08.553458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 496/983] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:08.558732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.571518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:08.574975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608579d3-1e41-49ca-b8af-b88c9295ed41 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.611370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.612288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.612722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.613343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.614106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.616225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.623041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 488/984] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:16:08.684036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d9b7f-e021-4178-84e2-12c67e09e7ae None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.722284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.723092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.723460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.723833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.735131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.736775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:08.753569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:08.754624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:08.760613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:08.760613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:08.760613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:08.760613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 497/985] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:08.760902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.870780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958a693-a4a2-4bf2-83c7-7c47b9a465f7 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:08.905842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:08.906631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:08.907013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:08.907389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:08.908136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:08.908541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:08.910634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:08.923119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 489/986] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:08.928552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:08.980797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:09.035378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eddbc9-80a4-4af4-8a02-3f88881e538e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:09.068208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:09.068951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:09.069300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:09.069643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:09.076629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:09.078171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:16:09.089796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:09.090863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:09.091979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:09.097691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 498/987] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:09.098780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:09.126181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:09.204280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461b67a-df7b-45ae-8948-47568bea328e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:09.238736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:09.239355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:09.239619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:09.239880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:09.240563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:09.240894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:09.242319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0f9f9-036f-4cea-b781-4130629104a6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:09.254557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 490/988] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:09.260289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:09.366754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9980217-4499-472c-9984-1dc838a72d83 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:09.378434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:09.401583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:09.402247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:09.402543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:09.402808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:09.403532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] RBAC: Adding query filter params (role.id=1ca160d1b03847e8a6058aa73a00705d, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=64622f7e541046d69f64a4393316abd7) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:09.403841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:09.405314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d82a-cbd4-41b9-a89b-4f0949c91331 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:09.416947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 499/989] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:09 2022] GET /identity/v3/role_assignments?user.id=64622f7e541046d69f64a4393316abd7&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=1ca160d1b03847e8a6058aa73a00705d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:10.336927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfe86a-5127-4d66-abac-bdcbfd9277ec None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:10.337503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfe86a-5127-4d66-abac-bdcbfd9277ec None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:10.337503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfe86a-5127-4d66-abac-bdcbfd9277ec None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:10.338966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 491/990] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:10.344236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fe7ac-935d-4b2c-8469-bf834739b09f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:10.344532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fe7ac-935d-4b2c-8469-bf834739b09f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:10.344817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fe7ac-935d-4b2c-8469-bf834739b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:10.363462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8b5fe7ac-935d-4b2c-8469-bf834739b09f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:10.364661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8b5fe7ac-935d-4b2c-8469-bf834739b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:10.435518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 500/991] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:10.714353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfe86a-5127-4d66-abac-bdcbfd9277ec None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:10.805592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:10.822756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfe86a-5127-4d66-abac-bdcbfd9277ec None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:10.850988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cede7-f3a3-412c-923b-dc7e72db7572 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:10.851366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cede7-f3a3-412c-923b-dc7e72db7572 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:10.851716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349cede7-f3a3-412c-923b-dc7e72db7572 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:10.853203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 501/992] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:10.858123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:10.858613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:10.859114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:10.861151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:10.861783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:10.862658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:10.862938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:10.863519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:10.864896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:10.873677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:10.874689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:10.875551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 492/993] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:10.880490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:10.880605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:10.881764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:10.953058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 502/994] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:11.012614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd71b6a-85ba-4910-b7d6-cadfb8dab57c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.046976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.047588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.047904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.048141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.048769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:11.049103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.051202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.060542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 493/995] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:11.062090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.097250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:11.163104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976687bd-37ca-494e-8267-83a24ad057a3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.198295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.198907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.199190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.199465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.200092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:11.200398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.201805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.207438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 503/996] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:11.213116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.227992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.229250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:11.323884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b8eb5c-fe0e-4d52-9433-38baa9ced97c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.329593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:11.334263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55ed77-d2d9-4523-ac94-13cebfd2c7cc None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.358916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.359557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.359865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.360142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.360839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] RBAC: Adding query filter params (interface=public, service_id=b9570a6fb031460e807b0de631600acc, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:11.361175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.363657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.373949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 494/997] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/endpoints?service_id=b9570a6fb031460e807b0de631600acc&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:11.374094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.374094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.374094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.374094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.374408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.374408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.375157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:11.384328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:11.385411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:11.386267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 504/998] 10.222.0.42 () {56 vars in 1126 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:11.391115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.456853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:11.483212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050fcd6-afe1-4a06-a406-30964b89e81f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.499227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f2147-2412-46ac-84e4-1e44b8146dcc None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.525355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.526380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.526876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.527362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.528187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.529790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.535256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.535867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.536150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.536421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.537080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:11.537389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.539524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80afcf-afe0-45aa-9223-8147ee5680a1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.541183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 495/999] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:16:11.544907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 505/1000] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:11.549982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.608148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58acca8-3629-4357-82bf-45e76ad76506 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.608485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58acca8-3629-4357-82bf-45e76ad76506 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.608697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58acca8-3629-4357-82bf-45e76ad76506 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.612370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 506/1001] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:11.616086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.616333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.617049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.639245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:11.640470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:11.664152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:11.670072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12534a1e-299b-4f7f-acad-c42829aa7ba6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.710367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.710884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.711162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.711429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.712050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:11.712350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.713736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.718444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 507/1002] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:11.724914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 496/1003] 10.222.0.42 () {56 vars in 1122 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:11.726669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.827093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b1df8-83f1-495d-8637-89fe43a7c3bb None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:11.860796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:11.861434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:11.861713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:11.862519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:11.863361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:11.864745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:11.943168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 508/1004] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:16:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:11.989463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:11.993635 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:12.103932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e7638-4afe-4ed2-ac10-5f9a3ca23c16 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:12.148282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:12.148933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:12.149217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:12.149489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] PATH_INFO: `/v3/services/image` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:12.150062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:12.151442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:12.161921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:12.162926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:12.163777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 497/1005] 10.222.0.42 () {56 vars in 1105 bytes} [Tue Oct 4 10:16:11 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:12.168929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:12.191484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:12.275311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6956f4d-c19c-4b0c-a585-496bb1743f8b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:12.310689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:12.311301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:12.311581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:12.311854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:12.312486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] RBAC: Adding query filter params (name=image) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:12.312789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:12.314200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc3bf5-1d48-46db-8f51-a7130ace23ab None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:12.326083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 509/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:12.327397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:12.393538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:12.433025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a0898-ac86-4876-8b77-68d54fd45b4a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:12.470776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:12.470776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:12.470776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:12.470776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:12.471801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] RBAC: Adding query filter params (type=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:12.471801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:12.473391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bdf99-186c-4272-a54f-d508a8f6770f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:12.481015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 498/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:13.778102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b90da3-8ea2-4b63-b05b-db7e83038459 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:13.778622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b90da3-8ea2-4b63-b05b-db7e83038459 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:13.778701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b90da3-8ea2-4b63-b05b-db7e83038459 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:13.780021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 510/1008] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:13.785874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa00026-dabb-4793-98c7-25c4159b108b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:13.786307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa00026-dabb-4793-98c7-25c4159b108b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:13.786386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa00026-dabb-4793-98c7-25c4159b108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:13.804587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-aaa00026-dabb-4793-98c7-25c4159b108b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:13.806133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aaa00026-dabb-4793-98c7-25c4159b108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:13.876007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 499/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:14.146403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b90da3-8ea2-4b63-b05b-db7e83038459 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:14.179232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909b0a-8fa7-4503-95f1-c4af77d39d0e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.179522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909b0a-8fa7-4503-95f1-c4af77d39d0e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.179794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b909b0a-8fa7-4503-95f1-c4af77d39d0e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.181264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 500/1010] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:14.186342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.186620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.186892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.207842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:14.209235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:14.235597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:14.256154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b90da3-8ea2-4b63-b05b-db7e83038459 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:14.270621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:14.288350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 501/1011] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:14.296804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:14.297489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.297770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.298046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.298661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:14.300408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:14.309862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:14.310909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:14.316995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 511/1012] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:14.319030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:14.420358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9e237-c7a1-4b97-9c1e-8e97187769e2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:14.454885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211081da-8143-4228-aebe-8bf0904460af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:14.455515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211081da-8143-4228-aebe-8bf0904460af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.455836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211081da-8143-4228-aebe-8bf0904460af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.456286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211081da-8143-4228-aebe-8bf0904460af None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.457093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211081da-8143-4228-aebe-8bf0904460af None admin] RBAC: Adding query filter params (name=compute) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:14.457480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211081da-8143-4228-aebe-8bf0904460af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:14.458660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211081da-8143-4228-aebe-8bf0904460af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:14.463841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 502/1013] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:14.469295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:14.553346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211081da-8143-4228-aebe-8bf0904460af None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:14.571592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbb26c-ae9b-43c9-bcfa-076728230e96 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:14.607521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:14.608128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.608405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.608679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.609367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] RBAC: Adding query filter params (type=compute) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:14.609667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:14.611034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:14.619982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 512/1014] 10.222.0.42 () {56 vars in 1112 bytes} [Tue Oct 4 10:16:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:14.622005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:14.625303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:14.636252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:14.676704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211081da-8143-4228-aebe-8bf0904460af None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:14.713490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f0955-83da-4b59-a70e-37d109061d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:14.714114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0955-83da-4b59-a70e-37d109061d70 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.714390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0955-83da-4b59-a70e-37d109061d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.714663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0955-83da-4b59-a70e-37d109061d70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.715753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f0955-83da-4b59-a70e-37d109061d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:14.717212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f0955-83da-4b59-a70e-37d109061d70 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:14.718781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 503/1015] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:14.719890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495522c-9d07-4751-9c7d-926f41858476 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.720169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495522c-9d07-4751-9c7d-926f41858476 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.720447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495522c-9d07-4751-9c7d-926f41858476 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.724895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 504/1016] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:14.725046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495522c-9d07-4751-9c7d-926f41858476 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:14.730374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26431f07-60bb-4d27-ae7c-e971cab037ca None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:14.764100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05df14-fe44-44cf-a96a-a6d857768087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:14.764705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05df14-fe44-44cf-a96a-a6d857768087 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.765016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05df14-fe44-44cf-a96a-a6d857768087 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.765292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05df14-fe44-44cf-a96a-a6d857768087 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.765960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05df14-fe44-44cf-a96a-a6d857768087 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:14.767324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05df14-fe44-44cf-a96a-a6d857768087 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:14.824116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495522c-9d07-4751-9c7d-926f41858476 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:14.844022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:14.852030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 513/1017] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:16:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:16:14.853276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.853561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.854022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.865968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a71219-ce41-42a4-86fe-9dd1a910c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:14.866595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71219-ce41-42a4-86fe-9dd1a910c8be None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:14.866871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71219-ce41-42a4-86fe-9dd1a910c8be None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:14.867144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a71219-ce41-42a4-86fe-9dd1a910c8be None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:14.867813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a71219-ce41-42a4-86fe-9dd1a910c8be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:14.869358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a71219-ce41-42a4-86fe-9dd1a910c8be None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:14.879204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:14.880474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:14.946183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:14.954877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 514/1018] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:14 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:14.968534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 505/1019] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:16:14 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:16:15.227688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:15.335178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d051a-e870-46d2-8893-bd78cd42b8a9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:15.369481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e034b84-a0b5-4b74-b584-7b2b0bb8c24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:15.370090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e034b84-a0b5-4b74-b584-7b2b0bb8c24f None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:15.370376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e034b84-a0b5-4b74-b584-7b2b0bb8c24f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:15.370652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e034b84-a0b5-4b74-b584-7b2b0bb8c24f None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:15.371309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e034b84-a0b5-4b74-b584-7b2b0bb8c24f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:15.372674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e034b84-a0b5-4b74-b584-7b2b0bb8c24f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:15.469779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 515/1020] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:16:15 2022] POST /identity/v3/services => generated 242 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:17.152176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422383e-25cb-4b06-9441-7617aedb16eb None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.152707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422383e-25cb-4b06-9441-7617aedb16eb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.152707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422383e-25cb-4b06-9441-7617aedb16eb None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.154354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 506/1021] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:17.156517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69c80c-0dfd-4d43-a224-7ad1881a54db None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.156926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69c80c-0dfd-4d43-a224-7ad1881a54db None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.157113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69c80c-0dfd-4d43-a224-7ad1881a54db None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.158440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 516/1022] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:17.160351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89547-4d47-443d-95e1-b0275163963e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.160640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89547-4d47-443d-95e1-b0275163963e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.160957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89547-4d47-443d-95e1-b0275163963e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.164277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.164552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.164821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.179774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-63f89547-4d47-443d-95e1-b0275163963e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:17.181077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-63f89547-4d47-443d-95e1-b0275163963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:17.182817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:17.183969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:17.234401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:17.260995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 517/1023] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:17.266801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 507/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:17.530009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:17.538177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89547-4d47-443d-95e1-b0275163963e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:17.642390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6097f50-bd7f-4cb4-83fc-0501695b25c0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:17.649096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:17.655798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89547-4d47-443d-95e1-b0275163963e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:17.679002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284542a-5d11-4d3f-b97f-2894b0ba09c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:17.679639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284542a-5d11-4d3f-b97f-2894b0ba09c8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.679919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284542a-5d11-4d3f-b97f-2894b0ba09c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.680195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284542a-5d11-4d3f-b97f-2894b0ba09c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.681398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284542a-5d11-4d3f-b97f-2894b0ba09c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:17.682846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284542a-5d11-4d3f-b97f-2894b0ba09c8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:17.684893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 518/1025] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:17.685556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c79146-0583-4b41-a917-3975fdcaf609 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.685840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c79146-0583-4b41-a917-3975fdcaf609 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.686123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c79146-0583-4b41-a917-3975fdcaf609 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.687560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 519/1026] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:17.690525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c79146-0583-4b41-a917-3975fdcaf609 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:17.693863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258a8c8-790e-4b12-9637-5cf3a477094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:17.694495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a8c8-790e-4b12-9637-5cf3a477094b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.694769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a8c8-790e-4b12-9637-5cf3a477094b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.695036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258a8c8-790e-4b12-9637-5cf3a477094b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.705189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258a8c8-790e-4b12-9637-5cf3a477094b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a258a8c8-790e-4b12-9637-5cf3a477094b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:17.710824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:17.712540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:17.713450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 508/1027] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Oct 4 10:16:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:17.714178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13483a25-570b-4c54-9958-0061302976af None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.714474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13483a25-570b-4c54-9958-0061302976af None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.714563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13483a25-570b-4c54-9958-0061302976af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.731393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-13483a25-570b-4c54-9958-0061302976af None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:17.733098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-13483a25-570b-4c54-9958-0061302976af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:17.743228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:17.794863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c79146-0583-4b41-a917-3975fdcaf609 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:17.798365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 509/1028] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:17.799741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13483a25-570b-4c54-9958-0061302976af None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:17.831129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:17.831738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.832025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.832305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.832984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:17.834419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:17.888312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:17.903805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13483a25-570b-4c54-9958-0061302976af None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:17.937560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 520/1029] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:16:17 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:17.943164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:17.943794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:17.944063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:17.944332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:17.945034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:17.945337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:17.946736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:17.957513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 510/1030] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:16:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:17.963344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.068101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.078480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dee012-1aa8-4419-8c29-9176fe68e8ea None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.081148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.118184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.118184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.119213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.119342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.132891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.134445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:18.150295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:18.152614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:18.153624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:18.153624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:18.153624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:18.156456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 521/1031] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:18.157362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.194144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57322d66-42c5-49bc-9893-b06c1304907b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.231497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.232127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.232407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.232678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.233370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.234726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:18.245082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:18.246098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:18.246943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:18.249743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 511/1032] 10.222.0.42 () {56 vars in 1105 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:18.255229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.269783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.279374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c837ae-e426-485f-b838-3a8a4230f7fe None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.317732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.318399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.318622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.318898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.319587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:18.319844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.321272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.336910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 522/1033] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:18.342511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.361487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.373694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5050e2-1228-421c-bb8e-fae935a273f4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.417073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.417688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.417965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.418228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.418865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:18.419168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.420528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.427221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 512/1034] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:18.432480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.453150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f531e-4c1f-4e32-9497-fba7c5535f58 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.491112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.491723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.492006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.492283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.499904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.501396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:16:18.513554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:18.514535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:18.517948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 523/1035] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:18.519277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.521253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.544246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c175c71-102c-4f4c-854d-a7110ec859a0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.582058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.582581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.582859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.583136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.583759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] RBAC: Adding query filter params (type=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:18.584059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.585469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.590404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 513/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:18.595899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.655104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.678864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca97ec-81ac-4ef4-b5f5-5a7c49d027f1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.716174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.716829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.717324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.717601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.718351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:18.718595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aabb59-463b-47ed-a0ef-63d3e0d94baf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.718698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.720183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.739383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 524/1037] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:18.744994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.754934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.755551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.755834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.756112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.756816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] RBAC: Adding query filter params (interface=public, service_id=670c542cc85e4343b111a7d3d924a044, region_id=RegionOne) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:18.757163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.758567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.763696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 514/1038] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/endpoints?service_id=670c542cc85e4343b111a7d3d924a044&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:18.770279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:18.772970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.857516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa07d0-990b-498e-a254-33129783aa0b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.883351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfe6b9-89bf-4e5f-bde5-95237c75ef6e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:18.894486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.895087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.895364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.895637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.898262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:18.925997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:18.927044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:18.927508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:18.927972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:18.928777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.930488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:18.937812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 515/1039] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:16:18.972036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:18.973551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:19.084226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 525/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 10:16:19.089554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:19.160541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:19.195439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcde863-1ac1-434a-bcfb-d6113e4d7513 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:19.230612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:19.231242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:19.231570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:19.231845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:19.243106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:19.244603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:19.259207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:19.260330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:19.261922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:19.261922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:19.261922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:19.261922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 516/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:19.266340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:19.300757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:19.368749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a2ba5-5e32-4904-948b-818c1955283d None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:19.403467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:19.404073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:19.404352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:19.404627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:19.405291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:19.405595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:19.406962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:19.423606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 526/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:19.425233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:19.526285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe059b-f836-4161-8d1b-5d29a3b10501 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:19.543531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:19.560210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:19.563394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:19.563394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:19.563394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:19.567898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:19.569378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:16:19.582960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:19.589145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:19.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 517/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:19.592753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:19.704773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:19.712030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a807a0-b446-4894-b58c-62d65d5dca06 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:19.746543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:19.747157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:19.747437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:19.747712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:19.748395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:19.748701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:19.750225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b49544-8aba-4a57-8380-6bd507959f64 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:19.767608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 527/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:19.769829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:19.873741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2f075-1d68-4378-94f1-4c025fc7040d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:19.907475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:19.908074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:19.908349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:19.909010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:19.909492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] RBAC: Adding query filter params (role.id=1ca160d1b03847e8a6058aa73a00705d, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=adc1b4c4a2de4d97bb8019c9efc05570) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:19.909797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:19.911204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4a2ea-ac17-476f-a812-9c506d925cf7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:19.916619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 518/1045] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:19 2022] GET /identity/v3/role_assignments?user.id=adc1b4c4a2de4d97bb8019c9efc05570&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=1ca160d1b03847e8a6058aa73a00705d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:20.244284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7caa-a439-44cc-b721-31d4417c6137 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:20.244668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7caa-a439-44cc-b721-31d4417c6137 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:20.244902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7caa-a439-44cc-b721-31d4417c6137 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:20.246351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 528/1046] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:20.251394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:20.251851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:20.251942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:20.271205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:20.272402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:20.308019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:20.340271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 519/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:20.625296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a7caa-a439-44cc-b721-31d4417c6137 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:20.729895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a7caa-a439-44cc-b721-31d4417c6137 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:20.764884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:20.765548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:20.765827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:20.766101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:20.778489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:20.783261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:20.785296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:20.786174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:20.789247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 529/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:20.791032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:20.890551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d6295-adef-48b8-8494-7350aed0bc4d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:20.925469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:20.926075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:20.926355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:20.926625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:20.927296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:20.927592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:20.928979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:20.942425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 520/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:20.943664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:20.961884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.052561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcba6af-a1db-4b02-b746-0893aa98a599 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.087325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93880433-31a2-401a-8802-e26714782f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.087966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93880433-31a2-401a-8802-e26714782f9a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.088247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93880433-31a2-401a-8802-e26714782f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.088521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93880433-31a2-401a-8802-e26714782f9a None admin] PATH_INFO: `/v3/users/b58b5ea45eff42698ef2cbef9b5659ab` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.102964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93880433-31a2-401a-8802-e26714782f9a None admin] RBAC: Authorizing `identity:get_user(user_id=b58b5ea45eff42698ef2cbef9b5659ab)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.104430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93880433-31a2-401a-8802-e26714782f9a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.106480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 530/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:20 2022] GET /identity/v3/users/b58b5ea45eff42698ef2cbef9b5659ab => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:21.112450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:21.115149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584ab6a-2b03-4f91-ad34-29178a5e750c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.115675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584ab6a-2b03-4f91-ad34-29178a5e750c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.115957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584ab6a-2b03-4f91-ad34-29178a5e750c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.118149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 531/1051] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:21.123422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.123710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.123987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.142102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:21.143845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:21.153167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.216424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 532/1052] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:21.231794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d3151-5f61-42d2-be10-03749208e8ca None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.266598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.267236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.267470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.267803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.335877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.337353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.369713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.440982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 521/1053] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:21.442727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:21.445348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.485973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:21.545075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa15438-2a9a-4123-89d8-46bdcb5529be None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.580187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.581517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.581806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.582094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.583221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] RBAC: Authorizing `identity:get_project(project_id=d9775d3fce7a47d58ac3b04d2475e660)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.584663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.586491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 533/1054] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:21 2022] GET /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:21.590223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00325a95-4596-43f5-9d4b-050f25eaf8fa None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.592259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:21.627831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.628563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.628833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.629135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] PATH_INFO: `/v3/services/image` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.629853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.631347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.633481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:21.641670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:21.642829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:21.643779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 522/1055] 10.222.0.42 () {56 vars in 1105 bytes} [Tue Oct 4 10:16:21 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:21.648741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:21.671460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.704220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada32d-2061-4e03-a3cf-b76a1229b68c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.740928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.741530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.741811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.742084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.742928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=d9775d3fce7a47d58ac3b04d2475e660, user.id=b58b5ea45eff42698ef2cbef9b5659ab) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:21.743243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.744667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.754590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e98b9-61a6-4d9d-81c5-8d40e03d7071 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.757253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 534/1056] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:21 2022] GET /identity/v3/role_assignments?user.id=b58b5ea45eff42698ef2cbef9b5659ab&scope.project.id=d9775d3fce7a47d58ac3b04d2475e660&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:21.794043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.794658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.794934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.795202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.795831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] RBAC: Adding query filter params (name=image) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:21.796129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.797539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.808905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 523/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:21.809924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:21.918068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565ed90-36b0-4a33-8e4f-35132d92de3e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:21.956169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:21.956696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:21.957002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:21.957274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:21.957922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] RBAC: Adding query filter params (type=image) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:21.958222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:21.959642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfdd6f5-3535-4fd1-be39-1b3104ce9079 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:21.961762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:21.972935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 535/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:21.975167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:22.043218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:22.067513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916dddd-8274-4a12-adfd-60df5ce1ed7f None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:22.067796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916dddd-8274-4a12-adfd-60df5ce1ed7f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:22.068070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916dddd-8274-4a12-adfd-60df5ce1ed7f None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:22.069599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 536/1059] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:22.074829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:22.075108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:22.075387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:22.088423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bde937-af7c-4b46-858a-41f85760c652 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:22.093889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:22.095140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:22.125521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:22.126131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:22.126418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:22.126687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:22.128043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:22.129638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:22.165262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 537/1060] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:22.218280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 524/1061] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:16:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:22.435041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:22.535415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77475928-b5dd-4556-aa2d-5e4c5d9822cd None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:22.564744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:22.578872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:22.579517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:22.579801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:22.580080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:22.591319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:22.596711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:22.597810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:22.598678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 538/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Oct 4 10:16:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:22.605392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:22.722867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c27ce-df84-451e-baf9-a1e3e8ceb9ae None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:22.740576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:22.757968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:22.758586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:22.758909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:22.759180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:22.759847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:22.760150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:22.761557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:22.771223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 525/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:16:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:22.776482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:22.876673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1451d-e7ab-438b-b542-020967fd5c71 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:22.911427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:22.912055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:22.912337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:22.916576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:22.922841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:22.924361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:22.938926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:22.940698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:22.941678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:22.941678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:22.941678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:22.941678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 539/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:22.948013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.054372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a14050-165b-43d2-a73b-cfc76daf9291 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:23.089154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:23.089764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.090050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.090322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.090960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:23.091263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:23.092620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:23.109035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 526/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:23.110760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.147781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:23.215184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14591b75-da57-4a21-9034-6aea7e45ed59 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:23.248149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:23.248769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.249088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.249345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.256976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:23.258396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:16:23.270342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:23.271298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:23.277326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 540/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:16:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:23.278022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.319941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:23.378903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ec4d9-c176-48bb-bbaf-e3e07f41ebfb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:23.415213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:23.416058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.416492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.416953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.417847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:23.418312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:23.419989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:23.443583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 527/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:16:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:23.444812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.541063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:23.547241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac4bd6-a3cf-41aa-97b9-1a87e2b2fe07 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:23.581294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:23.582432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.583070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.583605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.594000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:23.595435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:23.616614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:23.617689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:23.618636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:23.618636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:23.618636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:23.618636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 541/1068] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:23.618970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.716047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:23.720365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8ad27-3e7b-4cc2-996c-07dada6b25f1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:23.754172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:23.755008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.755286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.755555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.756186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:23.756482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:23.757894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ec055-94e9-4ce5-8a46-62225dc77706 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:23.769272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 528/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:23.774785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.879442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ad173-5656-428f-b98b-f71fd3bd61d6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:23.914220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:23.914851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.915128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.915398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.922296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:23.923745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:23.924462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd40e6-cc55-4f2b-b584-c80b24598112 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.924778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd40e6-cc55-4f2b-b584-c80b24598112 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.925127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd40e6-cc55-4f2b-b584-c80b24598112 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.926460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 529/1070] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:23.931707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:23.931982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:23.932254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:16:23.938266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:23.939354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:23.940364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 542/1071] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:23.945107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:23.954033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:23.955748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:23.957595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:24.027101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 530/1072] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:24.065021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45f870-374f-435f-bbfe-dd4de537182e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:24.099409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:24.100013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.100290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.100560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.101267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:24.101572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:24.102982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:24.120734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 543/1073] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:24.121605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:24.216673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:24.229269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7abeae-0829-473a-8196-2ae7caca6d28 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:24.262613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:24.263397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.263680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.263952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/adc1b4c4a2de4d97bb8019c9efc05570/roles/1ca160d1b03847e8a6058aa73a00705d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.266133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=adc1b4c4a2de4d97bb8019c9efc05570, role_id=1ca160d1b03847e8a6058aa73a00705d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:24.267973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:24.294848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:24.359500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:24.393799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 531/1074] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:16:24 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/adc1b4c4a2de4d97bb8019c9efc05570/roles/1ca160d1b03847e8a6058aa73a00705d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:16:24.399640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1bd6df-b4f1-4494-aca3-2f01f20e6d1c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:24.439391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48311803-a48c-4d1a-a07b-05f8498f7bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:24.440623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311803-a48c-4d1a-a07b-05f8498f7bfb None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.445756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311803-a48c-4d1a-a07b-05f8498f7bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.445756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311803-a48c-4d1a-a07b-05f8498f7bfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.453968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48311803-a48c-4d1a-a07b-05f8498f7bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-48311803-a48c-4d1a-a07b-05f8498f7bfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:24.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:24.461076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:24.462011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 544/1075] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:24.468890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:24.521339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31fee-38fb-42ca-82ea-38484dc866c0 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.521647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31fee-38fb-42ca-82ea-38484dc866c0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.521891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31fee-38fb-42ca-82ea-38484dc866c0 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.523248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 545/1076] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:24.530250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.530558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.530810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.553534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:24.554793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:24.593143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7533a28-85f7-43bd-a7c8-352b2669259d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:24.636309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 546/1077] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:24.639637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:24.640269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.640548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.640804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.641495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:24.641798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:24.643240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:24.652919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 532/1078] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:24.657592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:24.718036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:24.760978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3651c-9c5b-4aa5-88ad-e88352206aa9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:24.794803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:24.795430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.795704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.795975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] PATH_INFO: `/v3/users/b58b5ea45eff42698ef2cbef9b5659ab` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.797143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] RBAC: Authorizing `identity:get_user(user_id=b58b5ea45eff42698ef2cbef9b5659ab)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:24.798572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:24.800886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 547/1079] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:24 2022] GET /identity/v3/users/b58b5ea45eff42698ef2cbef9b5659ab => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:24.807120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:24.820326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:24.908069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700177-4118-4ad1-a2bf-ac4e596ba7f0 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:24.912513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:24.945004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:24.945640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:24.945915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:24.946181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:24.947296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] RBAC: Authorizing `identity:get_project(project_id=d9775d3fce7a47d58ac3b04d2475e660)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:24.949071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:24.950843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 533/1080] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:24 2022] GET /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:24.957057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:25.023620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436c5ad-3b74-4809-bf86-b57eda850a0e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:25.050519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:25.050892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:25.074583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:25.074583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:25.074583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:25.074583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:25.075044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae6573-aa87-4a11-b14c-44b8fe2d04af None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:25.107561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:25.108221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:25.108513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:25.108782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660/users/b58b5ea45eff42698ef2cbef9b5659ab/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:25.110755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9775d3fce7a47d58ac3b04d2475e660, user_id=b58b5ea45eff42698ef2cbef9b5659ab, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:25.112255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:25.131967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:25.133481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:25.195790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 534/1081] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:16:24 2022] PUT /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660/users/b58b5ea45eff42698ef2cbef9b5659ab/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:16:25.244205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 548/1082] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:25.250033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:25.367012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899cf38-740f-475b-b569-ba99ec835f34 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:25.405672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:25.406293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:25.406568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:25.406838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:25.413959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:25.417753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:25.419193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-50792843-b3d6-4aea-9557-423586ac00fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:25.433506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:25.434715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:25.435996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:25.435996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:25.435996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:25.435996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 535/1083] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:25.441346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:25.470082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:25.542043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd21d9-0e1f-48c5-95c0-0922bbc96018 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:25.577846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:25.578450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:25.578725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:25.578995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:25.579626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:25.579925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:25.581420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e080c9-039c-4f25-87f3-913c314183d2 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:25.600911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 549/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:26.619216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e79b1f-2716-4978-8876-7144e3728d54 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:26.619671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e79b1f-2716-4978-8876-7144e3728d54 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:26.619797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e79b1f-2716-4978-8876-7144e3728d54 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:26.621128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 536/1085] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:26.626341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:26.626751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:26.626894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:26.645949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:26.647122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:26.712814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 550/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:16:26.978168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e79b1f-2716-4978-8876-7144e3728d54 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.086417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e79b1f-2716-4978-8876-7144e3728d54 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.119108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.119729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.120009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.120281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.131167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:27.135899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:27.136910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:27.137785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 537/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Tue Oct 4 10:16:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:27.143270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.159998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:27.239521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e91d7-ce0e-4336-9de1-3c6e1abe4fa9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.271788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.272386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.272660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.272959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.273611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:27.273910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:27.275259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b58f3a-22fe-4bf7-a690-16801c60e15b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:27.282941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 551/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:27.289243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.341394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:27.392711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0883a0-b9d1-42db-8316-eb6272cca21c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.425535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.426155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.426429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.426692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.436822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:27.438319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:27.449190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd407-6d09-49f4-9916-17de065e46eb None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.449473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd407-6d09-49f4-9916-17de065e46eb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.449748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfd407-6d09-49f4-9916-17de065e46eb None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.451064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 552/1089] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:27.452785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:27.453937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:27.454867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:27.454867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:27.454867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:27.454867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 538/1090] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:27.456413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.456698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.457004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.459353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.475026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:27.476207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:27.542821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 553/1091] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:27.565637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c2aa4-9c41-4242-b76b-a92b739ebf11 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.567165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:27.599343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.599943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.600224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.600498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.601188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:27.601491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:27.602852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:27.619627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 539/1092] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:27.620822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.631350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:27.719126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784baf74-315d-4cde-b86f-1d588491dca9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.752455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.753163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.753449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.753727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.760543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:27.762020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:16:27.773878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:27.779552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:27.780694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 554/1093] 10.222.0.42 () {56 vars in 1093 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:27.781360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.813142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.878158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0800bc-65e8-4833-8e14-e6b1e6990f7e None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.879649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:27.912409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.912594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfd2df-fc6f-402c-a13e-6d0434ef7330 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:27.913047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.913334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.913610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.914333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:27.914648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:27.916072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:27.933380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 540/1094] 10.222.0.42 () {56 vars in 1133 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:27.946951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:27.947954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:27.948627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:27.948945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:27.949225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:27.960033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:27.962015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:27.965972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:27.967185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:27.968233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 555/1095] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:27.973092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.041272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da57dc-afdb-4a8d-8cbe-36edd57bc952 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.072149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1302b6-91a9-4125-ad5a-f3aab5d85a92 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.076614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.077252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.077538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.077808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.098979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.107318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.110048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.110048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.110048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.110048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:28.110048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.110976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.119928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 556/1096] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:28.125816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.146832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.148277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.168544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.223536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760a975-a3a6-4c89-a1cc-84c255cd397f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.246212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 541/1097] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 10:16:28.251866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.256815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.257483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.257763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.258040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] PATH_INFO: `/v3/users/b58b5ea45eff42698ef2cbef9b5659ab` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.259183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] RBAC: Authorizing `identity:get_user(user_id=b58b5ea45eff42698ef2cbef9b5659ab)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.260618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.266741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 557/1098] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/users/b58b5ea45eff42698ef2cbef9b5659ab => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:28.267817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.324252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.352438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637a7d0-98a4-4a6d-b41b-e934360bb974 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.367910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7174ca-fc0e-458b-9555-b19e2cd3269e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.385063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.402334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.402966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.403089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.403243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.403509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.403603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.403879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.404157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.413441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.414876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:16:28.430327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:28.431491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:16:28.432556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:16:28.432556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:16:28.432556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:28.432556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 542/1099] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:28.437174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.475960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.477435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.534890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.535089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da115dc-f914-4235-be7b-b92bf37e1be8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.569116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.569715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.569992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.570255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.570884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:28.571179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.572782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.576809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 558/1100] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:28.582767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.586354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 543/1101] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:28.592215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.625636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.684623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a2231-a8bb-43e1-a0f7-8610780d6796 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.704889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628296e-4f3a-4de7-8504-307fa9dd31bb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.730864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.731498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.731779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.732052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.733197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] RBAC: Authorizing `identity:get_project(project_id=d9775d3fce7a47d58ac3b04d2475e660)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.734636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.736330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 559/1102] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:28.739586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.740215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.740490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.740760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.742925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.747563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.749011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:16:28.761255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:28.762329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:16:28.763347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:28.767194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 544/1103] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:16:28.768618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:28.770320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.841851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d0c5c-7422-4b52-94d0-3a67143fa36a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.858157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:28.866895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee7455-4088-4cab-aa00-40b3920782c9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:28.875127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.875727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.876009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.876280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.877564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=d9775d3fce7a47d58ac3b04d2475e660, user.id=b58b5ea45eff42698ef2cbef9b5659ab) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:28.877867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.879395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.884982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 560/1104] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/role_assignments?user.id=b58b5ea45eff42698ef2cbef9b5659ab&scope.project.id=d9775d3fce7a47d58ac3b04d2475e660&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:16:28.902422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:28.903063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:28.903342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:28.903612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:28.904317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:28.904622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:28.906142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b144c63-227e-47a6-8803-6081b9d20cd6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:28.919279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 545/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:28.925744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:29.023224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:29.032904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc8d74-b503-4ead-889f-b9c9d7618158 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:29.071133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:29.071732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:29.072007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:29.072276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:29.073349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] RBAC: Adding query filter params (role.id=1ca160d1b03847e8a6058aa73a00705d, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=adc1b4c4a2de4d97bb8019c9efc05570) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:29.073653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:29.075076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75328b3e-0f38-4f4f-98ff-9f47206a7218 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:29.081439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 561/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:28 2022] GET /identity/v3/role_assignments?user.id=adc1b4c4a2de4d97bb8019c9efc05570&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=1ca160d1b03847e8a6058aa73a00705d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:31.086755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0cbf3-1b32-40e0-9998-b6fc8099b7d5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:31.087333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0cbf3-1b32-40e0-9998-b6fc8099b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:31.087333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0cbf3-1b32-40e0-9998-b6fc8099b7d5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:31.088642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 546/1107] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:31.093818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:31.094223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:31.094382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:31.112300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:31.113535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:31.177239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 562/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:31.450589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0cbf3-1b32-40e0-9998-b6fc8099b7d5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:31.552035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0cbf3-1b32-40e0-9998-b6fc8099b7d5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:31.585734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86592-7818-48f1-bb04-c04cdc2798c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:31.586346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86592-7818-48f1-bb04-c04cdc2798c9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:31.586625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86592-7818-48f1-bb04-c04cdc2798c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:31.586897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86592-7818-48f1-bb04-c04cdc2798c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:31.587985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86592-7818-48f1-bb04-c04cdc2798c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:31.589443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86592-7818-48f1-bb04-c04cdc2798c9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:31.590966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 547/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:31.596569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:31.678831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:31.694088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2f659-5686-4178-9a20-fe7b22942d6c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:31.727210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99f516-a9af-4bd2-a6a9-f0f50f5c50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:31.727746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99f516-a9af-4bd2-a6a9-f0f50f5c50a6 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:31.728028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99f516-a9af-4bd2-a6a9-f0f50f5c50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:31.728306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99f516-a9af-4bd2-a6a9-f0f50f5c50a6 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:31.728976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99f516-a9af-4bd2-a6a9-f0f50f5c50a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:31.730409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99f516-a9af-4bd2-a6a9-f0f50f5c50a6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:31.822262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 563/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:16:31 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:34.113244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd9f5-3457-4f98-94f2-dcbdd927d9d3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:34.113808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd9f5-3457-4f98-94f2-dcbdd927d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:34.113808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd9f5-3457-4f98-94f2-dcbdd927d9d3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:34.115217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 548/1111] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:16:34.121096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:34.121594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:34.121750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:34.140901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:34.142162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:34.213046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 564/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:34.501032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd9f5-3457-4f98-94f2-dcbdd927d9d3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:34.524326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:34.630918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd9f5-3457-4f98-94f2-dcbdd927d9d3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:34.667616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:34.668236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:34.668511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:34.668780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] PATH_INFO: `/v3/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:34.670167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] RBAC: Authorizing `identity:get_role(role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:34.671514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:34.672822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 549/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:34 2022] GET /identity/v3/roles/97b084644adb4949ab768912c0bb75e8 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:34.678865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:34.745299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:34.775437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14e35e-a061-400a-893f-68bd5917cb4d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:34.807685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:34.808303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:34.808578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:34.808845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] PATH_INFO: `/v3/users/e77771aa6b8e41b188f4336fbe7f2753` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:34.822194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] RBAC: Authorizing `identity:get_user(user_id=e77771aa6b8e41b188f4336fbe7f2753)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:34.823901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:34.825503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 565/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:34 2022] GET /identity/v3/users/e77771aa6b8e41b188f4336fbe7f2753 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:34.832225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:34.919549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:34.932031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731b186-3bc8-4bea-9da4-b5b71f1f3493 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:34.965169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:34.965774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:34.966055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:34.966329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:35.031528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:35.033730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:35.116632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:35.128622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 550/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:35.134823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:35.231054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440a13e-f8ea-4ed9-8d1c-0e8e864a5345 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:35.257381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:35.264885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c9376-6152-4aa5-89a5-6ebb8cef2c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:35.265509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9376-6152-4aa5-89a5-6ebb8cef2c1f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:35.268569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9376-6152-4aa5-89a5-6ebb8cef2c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:35.268569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9376-6152-4aa5-89a5-6ebb8cef2c1f None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:35.268569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c9376-6152-4aa5-89a5-6ebb8cef2c1f None admin] RBAC: Authorizing `identity:get_project(project_id=d9775d3fce7a47d58ac3b04d2475e660)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:35.268907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c9376-6152-4aa5-89a5-6ebb8cef2c1f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:35.270223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 566/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:35 2022] GET /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:35.276427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:35.372457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503dd5-aa9b-47bc-a8e6-8c01538acbd4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:35.404486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:35.405094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:35.405368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:35.405632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:35.406336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] RBAC: Adding query filter params (role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=d9775d3fce7a47d58ac3b04d2475e660, user.id=e77771aa6b8e41b188f4336fbe7f2753) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:35.406634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:35.408255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea47fc-4123-4463-9da1-83961ebc447d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:35.413376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 551/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:35 2022] GET /identity/v3/role_assignments?user.id=e77771aa6b8e41b188f4336fbe7f2753&scope.project.id=d9775d3fce7a47d58ac3b04d2475e660&role.id=97b084644adb4949ab768912c0bb75e8 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:37.515602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ee450-e54f-4b20-a75b-4486d422ee0d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:37.516162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ee450-e54f-4b20-a75b-4486d422ee0d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:37.516162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ee450-e54f-4b20-a75b-4486d422ee0d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:37.517588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 567/1118] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:37.522618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:37.523012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:37.523159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:37.541269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:37.542434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:37.588015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:37.606526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 552/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:37.885061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ee450-e54f-4b20-a75b-4486d422ee0d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:37.982836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ee450-e54f-4b20-a75b-4486d422ee0d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:38.015486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:38.016095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:38.016373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:38.016648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] PATH_INFO: `/v3/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:38.018264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] RBAC: Authorizing `identity:get_role(role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:38.019636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:38.020964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 568/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:37 2022] GET /identity/v3/roles/97b084644adb4949ab768912c0bb75e8 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:38.026723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:38.131970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945477a5-a870-4117-86d8-2b4408cdc982 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:38.165517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:38.166074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:38.166350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:38.166621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] PATH_INFO: `/v3/users/e77771aa6b8e41b188f4336fbe7f2753` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:38.167721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] RBAC: Authorizing `identity:get_user(user_id=e77771aa6b8e41b188f4336fbe7f2753)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:38.169171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:38.170742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 553/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:38 2022] GET /identity/v3/users/e77771aa6b8e41b188f4336fbe7f2753 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:38.176713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:38.237653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:38.273792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398faec3-3b4f-4e29-9000-9d0a1d6d2baf None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:38.306474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd624a1d-acd8-461b-9253-6f40c3f31f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:38.307089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd624a1d-acd8-461b-9253-6f40c3f31f08 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:38.307370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd624a1d-acd8-461b-9253-6f40c3f31f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:38.307647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd624a1d-acd8-461b-9253-6f40c3f31f08 None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:38.308753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd624a1d-acd8-461b-9253-6f40c3f31f08 None admin] RBAC: Authorizing `identity:get_project(project_id=d9775d3fce7a47d58ac3b04d2475e660)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:38.310212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd624a1d-acd8-461b-9253-6f40c3f31f08 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:38.311814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 569/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:38 2022] GET /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:38.317899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:38.367912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:38.416102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381be33-f45d-4291-a2c1-a320174e9a96 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:38.448953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885ee69-2539-4c50-9086-7e3663e41de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:38.449601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885ee69-2539-4c50-9086-7e3663e41de1 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:38.449881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885ee69-2539-4c50-9086-7e3663e41de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:38.450155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885ee69-2539-4c50-9086-7e3663e41de1 None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660/users/e77771aa6b8e41b188f4336fbe7f2753/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:38.452271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885ee69-2539-4c50-9086-7e3663e41de1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9775d3fce7a47d58ac3b04d2475e660, user_id=e77771aa6b8e41b188f4336fbe7f2753, role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:38.453723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885ee69-2539-4c50-9086-7e3663e41de1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:38.541593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 554/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:16:38 2022] PUT /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660/users/e77771aa6b8e41b188f4336fbe7f2753/roles/97b084644adb4949ab768912c0bb75e8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:16:40.685227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258a823-30f6-41de-bd7c-164e1700e012 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:40.685776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258a823-30f6-41de-bd7c-164e1700e012 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:40.685776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258a823-30f6-41de-bd7c-164e1700e012 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:40.687093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 570/1124] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:40.693112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:40.693508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:40.693666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:40.712946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:40.714173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:40.784938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 555/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:41.058890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258a823-30f6-41de-bd7c-164e1700e012 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:41.164289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258a823-30f6-41de-bd7c-164e1700e012 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:41.193295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:41.198251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:41.198873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:41.199153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:41.199428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] PATH_INFO: `/v3/roles/97b084644adb4949ab768912c0bb75e8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:41.200823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] RBAC: Authorizing `identity:get_role(role_id=97b084644adb4949ab768912c0bb75e8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:41.202213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:41.203517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 571/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:41 2022] GET /identity/v3/roles/97b084644adb4949ab768912c0bb75e8 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:41.209880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:41.249368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:41.303906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a9cd-3b7f-466d-b167-7120600d60c2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:41.336973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:41.337534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:41.337810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:41.338084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] PATH_INFO: `/v3/users/e77771aa6b8e41b188f4336fbe7f2753` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:41.339174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] RBAC: Authorizing `identity:get_user(user_id=e77771aa6b8e41b188f4336fbe7f2753)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:41.340588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:41.342161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 556/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:41 2022] GET /identity/v3/users/e77771aa6b8e41b188f4336fbe7f2753 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:41.348436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:41.445377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078543a4-acc5-4db3-a16e-aacac56f0ca6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:41.477645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:41.478236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:41.478510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:41.478780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:41.540452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:41.542590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:41.544037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:41.638649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 572/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:41.645108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:41.740801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc0acd-7d2f-4abd-a2c1-0bd99d07c494 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:41.749608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:41.773661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93339608-c67a-4f0c-bb83-ac25d80b1249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:41.774272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93339608-c67a-4f0c-bb83-ac25d80b1249 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:41.774565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93339608-c67a-4f0c-bb83-ac25d80b1249 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:41.774812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93339608-c67a-4f0c-bb83-ac25d80b1249 None admin] PATH_INFO: `/v3/projects/d9775d3fce7a47d58ac3b04d2475e660` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:41.775905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93339608-c67a-4f0c-bb83-ac25d80b1249 None admin] RBAC: Authorizing `identity:get_project(project_id=d9775d3fce7a47d58ac3b04d2475e660)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:41.777352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93339608-c67a-4f0c-bb83-ac25d80b1249 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:41.778936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 557/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:41 2022] GET /identity/v3/projects/d9775d3fce7a47d58ac3b04d2475e660 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:41.785675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:41.882184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:41.885262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7695ee-1056-4397-a357-4813d4aba50a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:41.918164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:41.918764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:41.919046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:41.919323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:41.920086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] RBAC: Adding query filter params (role.id=97b084644adb4949ab768912c0bb75e8, scope.project.id=d9775d3fce7a47d58ac3b04d2475e660, user.id=e77771aa6b8e41b188f4336fbe7f2753) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:41.920394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:41.921861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4072fe-4008-4c09-8039-0dbf83b3733f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:41.927861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 573/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:41 2022] GET /identity/v3/role_assignments?user.id=e77771aa6b8e41b188f4336fbe7f2753&scope.project.id=d9775d3fce7a47d58ac3b04d2475e660&role.id=97b084644adb4949ab768912c0bb75e8 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:44.080721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441313b-8c08-430a-a290-f2684188c56a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:44.081330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441313b-8c08-430a-a290-f2684188c56a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:44.081330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441313b-8c08-430a-a290-f2684188c56a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:44.082624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 558/1131] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:44.087720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:44.088099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:44.088269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:44.106878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:44.108052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:44.172913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 574/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:44.441579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441313b-8c08-430a-a290-f2684188c56a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:44.541462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441313b-8c08-430a-a290-f2684188c56a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:44.574380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda6078-64af-45f1-8001-61f619f8b05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:44.574991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6078-64af-45f1-8001-61f619f8b05d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:44.575269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6078-64af-45f1-8001-61f619f8b05d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:44.575542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda6078-64af-45f1-8001-61f619f8b05d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:44.576646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda6078-64af-45f1-8001-61f619f8b05d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:44.578108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda6078-64af-45f1-8001-61f619f8b05d None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:44.580375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 559/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:44.586280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:44.675103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:44.681515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e58e9-f9db-4a57-a661-b9a4849f66ee None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:44.716245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca8662-3936-4f68-baaa-28a9e5210396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:44.716761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca8662-3936-4f68-baaa-28a9e5210396 None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:44.717071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca8662-3936-4f68-baaa-28a9e5210396 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:44.717344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca8662-3936-4f68-baaa-28a9e5210396 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:44.717979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca8662-3936-4f68-baaa-28a9e5210396 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:44.719401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca8662-3936-4f68-baaa-28a9e5210396 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:44.838794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 575/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:16:44 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:47.062579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae945ad-122d-4332-8985-5a34a341b275 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:47.063105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae945ad-122d-4332-8985-5a34a341b275 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:47.063105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae945ad-122d-4332-8985-5a34a341b275 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:47.064445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 560/1135] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:47.070109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:47.070561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:47.070658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:47.090542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:47.091751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:47.155796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 576/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:47.431586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae945ad-122d-4332-8985-5a34a341b275 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:47.477604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:47.530273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae945ad-122d-4332-8985-5a34a341b275 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:47.562376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fa9f0-44db-4d87-a790-5ae06194ab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:47.563000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fa9f0-44db-4d87-a790-5ae06194ab97 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:47.563280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fa9f0-44db-4d87-a790-5ae06194ab97 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:47.563554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fa9f0-44db-4d87-a790-5ae06194ab97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:47.565430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fa9f0-44db-4d87-a790-5ae06194ab97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:47.566925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fa9f0-44db-4d87-a790-5ae06194ab97 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:47.568481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 561/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:47.574394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:47.650341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:47.672095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b72a0e-a0ac-4a8e-ac1d-daa6c4048968 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:47.706774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1217b-085b-4ecc-93b6-e90e615ca0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:47.707320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1217b-085b-4ecc-93b6-e90e615ca0bc None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:47.707602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1217b-085b-4ecc-93b6-e90e615ca0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:47.707877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1217b-085b-4ecc-93b6-e90e615ca0bc None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:47.708533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1217b-085b-4ecc-93b6-e90e615ca0bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:47.709996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1217b-085b-4ecc-93b6-e90e615ca0bc None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:47.800023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 577/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:16:47 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:16:50.017744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23354f-b26f-4674-af88-d1dc636b72fa None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:50.018283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23354f-b26f-4674-af88-d1dc636b72fa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:50.018283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23354f-b26f-4674-af88-d1dc636b72fa None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:50.019603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 562/1139] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:50.024786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:50.025219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:50.025373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:50.043757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:50.044980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:50.110117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 578/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:50.381712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23354f-b26f-4674-af88-d1dc636b72fa None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:50.479388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23354f-b26f-4674-af88-d1dc636b72fa None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:50.491746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:50.511603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:50.512214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:50.512490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:50.512757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:50.523102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:50.528145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:50.529405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:50.530541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 563/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:50.535588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:50.593612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:50.638004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183fc7-924c-484a-8945-dd0d307bdd11 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:50.665207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:50.665799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:50.666073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:50.666341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:50.666999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:50.667472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:50.669222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:50.678158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 579/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:50.684123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:50.784542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184a0f-eb3e-4330-8a98-7055ed1c7bb1 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:50.817292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:50.817913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:50.818194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:50.818473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] PATH_INFO: `/v3/users/18ed062a5bd648d290777ccf2cd64852` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:50.832400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] RBAC: Authorizing `identity:get_user(user_id=18ed062a5bd648d290777ccf2cd64852)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:50.833874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:50.835499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 564/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:50 2022] GET /identity/v3/users/18ed062a5bd648d290777ccf2cd64852 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:50.841383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:50.936615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae94f1e-8cdd-4b22-96da-7bee2db043ef None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:50.959876 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:50.970515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:50.978391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:50.978391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:50.978391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:51.035239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:51.036662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:51.130837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 580/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:51.136881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:51.167350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:51.235473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496f10-ca6d-430e-99f1-0e810eb39eff None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:51.268753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554cfa7-346c-4596-8358-00d6e5f76ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:51.269399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554cfa7-346c-4596-8358-00d6e5f76ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:51.269677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554cfa7-346c-4596-8358-00d6e5f76ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:51.269955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554cfa7-346c-4596-8358-00d6e5f76ea7 None admin] PATH_INFO: `/v3/projects/e7f0ed31333b4abeac338c65f5934dbd` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:51.271054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554cfa7-346c-4596-8358-00d6e5f76ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=e7f0ed31333b4abeac338c65f5934dbd)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:51.272478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554cfa7-346c-4596-8358-00d6e5f76ea7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:51.274102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 565/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:51 2022] GET /identity/v3/projects/e7f0ed31333b4abeac338c65f5934dbd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:51.280265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:51.303517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:51.377655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a0ecd-ebfe-4bbb-804a-5fb645d284ae None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:51.410733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:51.411328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:51.411607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:51.411880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:51.412594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=e7f0ed31333b4abeac338c65f5934dbd, user.id=18ed062a5bd648d290777ccf2cd64852) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:51.412945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:51.414360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3563b-99b1-4d6f-9c84-772ac9b6829b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:51.419526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 581/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:51 2022] GET /identity/v3/role_assignments?user.id=18ed062a5bd648d290777ccf2cd64852&scope.project.id=e7f0ed31333b4abeac338c65f5934dbd&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:53.506230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f99e14-24ae-4d0a-9bbe-1caee2a4e73b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:53.506808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f99e14-24ae-4d0a-9bbe-1caee2a4e73b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:53.506808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f99e14-24ae-4d0a-9bbe-1caee2a4e73b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:53.508100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 566/1147] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:53.513102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:53.513526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:53.513635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:53.531903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:53.533832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:53.598069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 582/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:53.863073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f99e14-24ae-4d0a-9bbe-1caee2a4e73b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:53.957113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:53.961417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f99e14-24ae-4d0a-9bbe-1caee2a4e73b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:53.994528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:53.995139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:53.995416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:53.995687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:54.005447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:54.010127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:54.011245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:16:54.012278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 567/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:16:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:54.017668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:54.035651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:54.113365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bffc14-62f2-497e-b734-30ce9af7bbb1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:54.145759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:54.146345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:54.146621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:54.146891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:54.147536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:54.147834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:54.149212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:54.156826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 583/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:54.163092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:54.258716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f0cd4-476f-411d-98e4-12d14212ca2d None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:54.290519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:54.291125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:54.291402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:54.291669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] PATH_INFO: `/v3/users/18ed062a5bd648d290777ccf2cd64852` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:54.292788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] RBAC: Authorizing `identity:get_user(user_id=18ed062a5bd648d290777ccf2cd64852)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:54.294232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:54.295758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 568/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:54 2022] GET /identity/v3/users/18ed062a5bd648d290777ccf2cd64852 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:54.301718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:54.379348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:54.397621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b10f1-54a9-480f-8b7c-f76a7b9e80b5 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:54.429565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056cd01-19ef-4f42-a47a-b401115c060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:54.430156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056cd01-19ef-4f42-a47a-b401115c060a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:54.430432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056cd01-19ef-4f42-a47a-b401115c060a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:54.430702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056cd01-19ef-4f42-a47a-b401115c060a None admin] PATH_INFO: `/v3/projects/e7f0ed31333b4abeac338c65f5934dbd` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:54.431792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056cd01-19ef-4f42-a47a-b401115c060a None admin] RBAC: Authorizing `identity:get_project(project_id=e7f0ed31333b4abeac338c65f5934dbd)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:54.433235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056cd01-19ef-4f42-a47a-b401115c060a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:54.434808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 584/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:54 2022] GET /identity/v3/projects/e7f0ed31333b4abeac338c65f5934dbd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:54.441503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:54.469162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:54.538417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7423e6-65d3-41a2-868b-9329030631db None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:54.571101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddbe0e-f7d1-46a0-a428-187a02e9a361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:54.571739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddbe0e-f7d1-46a0-a428-187a02e9a361 None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:54.572015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddbe0e-f7d1-46a0-a428-187a02e9a361 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:54.572282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddbe0e-f7d1-46a0-a428-187a02e9a361 None admin] PATH_INFO: `/v3/projects/e7f0ed31333b4abeac338c65f5934dbd/users/18ed062a5bd648d290777ccf2cd64852/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:54.574427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddbe0e-f7d1-46a0-a428-187a02e9a361 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7f0ed31333b4abeac338c65f5934dbd, user_id=18ed062a5bd648d290777ccf2cd64852, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:54.575841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ddbe0e-f7d1-46a0-a428-187a02e9a361 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:54.662342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 569/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:16:54 2022] PUT /identity/v3/projects/e7f0ed31333b4abeac338c65f5934dbd/users/18ed062a5bd648d290777ccf2cd64852/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:16:56.826490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611665d0-dbd0-48fd-81c0-6f86f490f03c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:56.827016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611665d0-dbd0-48fd-81c0-6f86f490f03c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:56.827016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611665d0-dbd0-48fd-81c0-6f86f490f03c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:56.828379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 585/1154] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:16:56.834447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:56.834879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:56.834992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:56.853290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:16:56.854506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:16:56.919229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 570/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:16:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:16:57.189318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611665d0-dbd0-48fd-81c0-6f86f490f03c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:57.288285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611665d0-dbd0-48fd-81c0-6f86f490f03c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:57.321701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:57.322312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:57.322588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:57.323112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:57.332852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:16:57.337598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:16:57.338698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:16:57.339758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 586/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:16:57.345319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:57.362716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:57.441634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf2367-92d0-471f-8f68-b3f7cc6b47e1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:57.475128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:57.475720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:57.476000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:57.476274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:57.476958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:57.477263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:57.478659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:57.487116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 571/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:57.492963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:57.503924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:57.589955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6ce5c-48ec-4e10-83ed-cadbd0a1ca3c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:57.624279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:57.624930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:57.625211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:57.625481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] PATH_INFO: `/v3/users/18ed062a5bd648d290777ccf2cd64852` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:57.626900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] RBAC: Authorizing `identity:get_user(user_id=18ed062a5bd648d290777ccf2cd64852)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:57.628321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:57.629902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 587/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:16:57 2022] GET /identity/v3/users/18ed062a5bd648d290777ccf2cd64852 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:57.635639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:57.714286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:57.736107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbad7d-0241-4dec-b5ec-9a073d5684eb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:57.769485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:57.770070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:57.770345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:57.770614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:57.836460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:57.837928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:57.931480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:57.953266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 572/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:16:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:16:57.960033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:58.033673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:16:58.055027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec5eaf-57e0-446c-860e-7ccfb4f5df81 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:58.087992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad887c-b4b7-41d0-a7b1-6561327d57f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:58.088598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad887c-b4b7-41d0-a7b1-6561327d57f8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:58.088891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad887c-b4b7-41d0-a7b1-6561327d57f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:58.089165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad887c-b4b7-41d0-a7b1-6561327d57f8 None admin] PATH_INFO: `/v3/projects/e7f0ed31333b4abeac338c65f5934dbd` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:58.090250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad887c-b4b7-41d0-a7b1-6561327d57f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e7f0ed31333b4abeac338c65f5934dbd)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:58.091660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad887c-b4b7-41d0-a7b1-6561327d57f8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:58.093266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 588/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:16:57 2022] GET /identity/v3/projects/e7f0ed31333b4abeac338c65f5934dbd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:16:58.099305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:16:58.194039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f5f28-b5ee-4946-83f1-8eb92289f30c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:16:58.228510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:16:58.229130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:16:58.229404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:16:58.229669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:16:58.230383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=e7f0ed31333b4abeac338c65f5934dbd, user.id=18ed062a5bd648d290777ccf2cd64852) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:16:58.230684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:16:58.232102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36878b-df1e-4969-a9ac-48bc4f474364 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:16:58.238817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 573/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:16:58 2022] GET /identity/v3/role_assignments?user.id=18ed062a5bd648d290777ccf2cd64852&scope.project.id=e7f0ed31333b4abeac338c65f5934dbd&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:00.383157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcf261-10cb-4791-aacf-1293e97a9332 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:00.383738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcf261-10cb-4791-aacf-1293e97a9332 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:00.383738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcf261-10cb-4791-aacf-1293e97a9332 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:00.385128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 589/1162] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:00.390241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:00.390659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:00.390787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:00.409105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:00.410279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:00.443594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:00.473829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 574/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:00.748727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcf261-10cb-4791-aacf-1293e97a9332 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:00.852916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcf261-10cb-4791-aacf-1293e97a9332 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:00.885501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:00.886115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:00.886398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:00.886675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:00.901463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:00.955779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:00.957244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.051975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 590/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:17:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:17:01.057937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.153709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9e53b-86e3-45b3-922d-8c531312df42 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.185977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.186597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.186877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.187150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.194342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.195782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-67ac7c8a-5cdc-4ae3-9f9f-99a8c770b6d5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:17:01.207645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:17:01.208930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:17:01.209823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 575/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:17:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:17:01.214597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.259122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:01.314737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a596c458-56d0-466a-89ef-73f358d0c4c4 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.348074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.348660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.348978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.349251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.349876 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:01.350173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.351520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57402df9-58af-45a4-abc0-983cd8a2bf75 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.358324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 591/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Tue Oct 4 10:17:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:03.532931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86424c-b170-41a9-b73c-4fcde092019b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:03.533941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86424c-b170-41a9-b73c-4fcde092019b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:03.533941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86424c-b170-41a9-b73c-4fcde092019b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:03.534998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 576/1167] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:03.541758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47babc-7cef-4879-b3e8-c2140a4219da None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:03.542230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47babc-7cef-4879-b3e8-c2140a4219da None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:03.542327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47babc-7cef-4879-b3e8-c2140a4219da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:03.561046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5f47babc-7cef-4879-b3e8-c2140a4219da None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:03.562219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5f47babc-7cef-4879-b3e8-c2140a4219da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:03.628394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 592/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:03.896941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86424c-b170-41a9-b73c-4fcde092019b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:03.932598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:04.001888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86424c-b170-41a9-b73c-4fcde092019b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:04.033395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ada33-1cc9-4603-90a0-805d155291d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:04.034519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ada33-1cc9-4603-90a0-805d155291d7 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:04.034799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ada33-1cc9-4603-90a0-805d155291d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:04.035078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ada33-1cc9-4603-90a0-805d155291d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:04.035742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ada33-1cc9-4603-90a0-805d155291d7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:04.037121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ada33-1cc9-4603-90a0-805d155291d7 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:04.125695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 577/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Tue Oct 4 10:17:03 2022] POST /identity/v3/domains => generated 267 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:17:06.294574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c12fa-fdc6-418e-be77-dcf5e2aa95c7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:06.295117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c12fa-fdc6-418e-be77-dcf5e2aa95c7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:06.295117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c12fa-fdc6-418e-be77-dcf5e2aa95c7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:06.296468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 593/1170] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:06.302845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:06.303230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:06.303399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:06.321991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:06.323423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:06.387191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 578/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:06.654714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c12fa-fdc6-418e-be77-dcf5e2aa95c7 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:06.696257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:06.755788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c12fa-fdc6-418e-be77-dcf5e2aa95c7 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:06.788461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd63a0b-e6d1-41b5-b9bf-0f2d6e40838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:06.789101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd63a0b-e6d1-41b5-b9bf-0f2d6e40838f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:06.789382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd63a0b-e6d1-41b5-b9bf-0f2d6e40838f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:06.789659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd63a0b-e6d1-41b5-b9bf-0f2d6e40838f None admin] PATH_INFO: `/v3/domains/b3c7ca7b2cc2472bbe47a4291c5ae511` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:06.800412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd63a0b-e6d1-41b5-b9bf-0f2d6e40838f None admin] RBAC: Authorizing `identity:get_domain(domain_id=b3c7ca7b2cc2472bbe47a4291c5ae511)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:06.802253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd63a0b-e6d1-41b5-b9bf-0f2d6e40838f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:06.805037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 594/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Tue Oct 4 10:17:06 2022] GET /identity/v3/domains/b3c7ca7b2cc2472bbe47a4291c5ae511 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:06.810729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:06.834344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:06.906455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e9939-2b82-455e-a229-74ae22f4a6a2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:06.939207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8097c0-183b-4e54-929a-922a5be828fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:06.940410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8097c0-183b-4e54-929a-922a5be828fe None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:06.940410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8097c0-183b-4e54-929a-922a5be828fe None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:06.940410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8097c0-183b-4e54-929a-922a5be828fe None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:06.941055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8097c0-183b-4e54-929a-922a5be828fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:06.942455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8097c0-183b-4e54-929a-922a5be828fe None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:07.085303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 579/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:17:06 2022] POST /identity/v3/projects => generated 361 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:17:09.243271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6826ce2-b3ea-4f5c-8c32-2d144828ca3f None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:09.243812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6826ce2-b3ea-4f5c-8c32-2d144828ca3f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:09.243812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6826ce2-b3ea-4f5c-8c32-2d144828ca3f None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:09.245182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 595/1174] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:09.250325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:09.250707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:09.250875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:09.269911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:09.271082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:09.334771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 580/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:09.598165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6826ce2-b3ea-4f5c-8c32-2d144828ca3f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:09.693608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:09.698472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6826ce2-b3ea-4f5c-8c32-2d144828ca3f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:09.730171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840cf7e-3f4b-48ba-9398-6a4581d6aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:09.731001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840cf7e-3f4b-48ba-9398-6a4581d6aa69 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:09.731331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840cf7e-3f4b-48ba-9398-6a4581d6aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:09.731909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840cf7e-3f4b-48ba-9398-6a4581d6aa69 None admin] PATH_INFO: `/v3/domains/b3c7ca7b2cc2472bbe47a4291c5ae511` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:09.733072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840cf7e-3f4b-48ba-9398-6a4581d6aa69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b3c7ca7b2cc2472bbe47a4291c5ae511)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:09.734504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840cf7e-3f4b-48ba-9398-6a4581d6aa69 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:09.736041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 596/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Tue Oct 4 10:17:09 2022] GET /identity/v3/domains/b3c7ca7b2cc2472bbe47a4291c5ae511 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:09.742478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:09.745533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:09.838763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b802d-cf84-4dd6-b77d-7245e300f168 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:09.871302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578910e5-fb53-4f96-8db8-7d3719db9267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:09.871846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578910e5-fb53-4f96-8db8-7d3719db9267 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:09.872125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578910e5-fb53-4f96-8db8-7d3719db9267 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:09.872397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578910e5-fb53-4f96-8db8-7d3719db9267 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:09.873086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578910e5-fb53-4f96-8db8-7d3719db9267 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:09.874508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578910e5-fb53-4f96-8db8-7d3719db9267 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:09.970042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 581/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:17:09 2022] POST /identity/v3/users => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:17:12.186949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b505-fc37-40f4-89d4-5ea0165feff5 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:12.187512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b505-fc37-40f4-89d4-5ea0165feff5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:12.187512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b505-fc37-40f4-89d4-5ea0165feff5 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:12.188832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 597/1178] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:12.194168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:12.194548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:12.194715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:12.212701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:12.213923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:12.268784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:12.279460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 582/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:12.551029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744b505-fc37-40f4-89d4-5ea0165feff5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:12.651337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744b505-fc37-40f4-89d4-5ea0165feff5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:12.684255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:12.684897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:12.685177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:12.685451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:12.695894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:17:12.700883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:17:12.702018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:17:12.703079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 598/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:17:12.708142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:12.808192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d83d0-003c-467d-bece-ae1a364dba9f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:12.841101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:12.841697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:12.841977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:12.842251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:12.842920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:12.843223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:12.844588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:12.855636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 583/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:17:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:12.862758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:12.890103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:12.960136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dd82a-d9d4-4f20-bea7-ac02502a3b30 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:12.993989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:12.994605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:12.994880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:12.995149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] PATH_INFO: `/v3/users/8656ec8244ac4ec688951c2ad60e3904` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:13.007751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] RBAC: Authorizing `identity:get_user(user_id=8656ec8244ac4ec688951c2ad60e3904)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:13.009224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:13.011557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 599/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:17:12 2022] GET /identity/v3/users/8656ec8244ac4ec688951c2ad60e3904 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:13.016662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:13.068588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:13.113549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24015a33-0f2c-4cb8-8241-f329b2611cc9 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:13.145724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:13.146313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:13.146588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:13.146855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:13.210688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:13.212104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:13.263312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:13.309747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 584/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:17:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:17:13.316065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:13.397185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:13.412522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f1de6-6022-4a3e-b14b-275575c1ca2a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:13.444715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f806056-14c8-43a6-ad6b-ab33ec59e859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:13.445348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f806056-14c8-43a6-ad6b-ab33ec59e859 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:13.445622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f806056-14c8-43a6-ad6b-ab33ec59e859 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:13.446034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f806056-14c8-43a6-ad6b-ab33ec59e859 None admin] PATH_INFO: `/v3/projects/98517b32cdd846b8863aae632875ea20` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:13.447146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f806056-14c8-43a6-ad6b-ab33ec59e859 None admin] RBAC: Authorizing `identity:get_project(project_id=98517b32cdd846b8863aae632875ea20)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:13.448839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f806056-14c8-43a6-ad6b-ab33ec59e859 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:13.450452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 600/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:17:13 2022] GET /identity/v3/projects/98517b32cdd846b8863aae632875ea20 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:13.457190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:13.556898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21430eea-0845-4448-b375-600f7459ed2a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:13.590396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:13.590995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:13.591276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:13.591550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:13.592265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=98517b32cdd846b8863aae632875ea20, user.id=8656ec8244ac4ec688951c2ad60e3904) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:13.592573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:13.594035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f2f53-653c-4aff-bd4e-b4351a93a081 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:13.599883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 585/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:17:13 2022] GET /identity/v3/role_assignments?user.id=8656ec8244ac4ec688951c2ad60e3904&scope.project.id=98517b32cdd846b8863aae632875ea20&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:15.686337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbcc29-f031-4b04-a12f-394fec01e835 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:15.686831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbcc29-f031-4b04-a12f-394fec01e835 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:15.686951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbcc29-f031-4b04-a12f-394fec01e835 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:15.688229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 601/1186] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:15.693961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:15.694367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:15.694501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:15.712449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:15.713650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:15.723935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:15.777573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 586/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:16.044115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbcc29-f031-4b04-a12f-394fec01e835 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:16.144039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bbcc29-f031-4b04-a12f-394fec01e835 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:16.177112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:16.177719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:16.177995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:16.178265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:16.187983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:17:16.192621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:17:16.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:17:16.195022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 602/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:17:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:17:16.199953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:16.297443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6dfa-3a7b-4498-bcd7-45234f97d182 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:16.332247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:16.332840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:16.333146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:16.333417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:16.334068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:16.334368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:16.335721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:16.343416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 587/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:16.350003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:16.364627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:16.446987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d0eaa-4bef-48ea-a838-c0eaf068aadf None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:16.480128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:16.480737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:16.481038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:16.481307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] PATH_INFO: `/v3/users/8656ec8244ac4ec688951c2ad60e3904` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:16.482397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] RBAC: Authorizing `identity:get_user(user_id=8656ec8244ac4ec688951c2ad60e3904)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:16.483816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:16.485388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 603/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:17:16 2022] GET /identity/v3/users/8656ec8244ac4ec688951c2ad60e3904 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:16.491203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:16.502444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:16.588585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627243ea-1f42-4067-b50a-590e0a664352 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:16.621676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89fed3-4fc7-41ad-8f38-6bcd3d9dc53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:16.622298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89fed3-4fc7-41ad-8f38-6bcd3d9dc53f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:16.622572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89fed3-4fc7-41ad-8f38-6bcd3d9dc53f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:16.622841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89fed3-4fc7-41ad-8f38-6bcd3d9dc53f None admin] PATH_INFO: `/v3/projects/98517b32cdd846b8863aae632875ea20` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:16.623932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89fed3-4fc7-41ad-8f38-6bcd3d9dc53f None admin] RBAC: Authorizing `identity:get_project(project_id=98517b32cdd846b8863aae632875ea20)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:16.625378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89fed3-4fc7-41ad-8f38-6bcd3d9dc53f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:16.627230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 588/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:17:16 2022] GET /identity/v3/projects/98517b32cdd846b8863aae632875ea20 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:16.633118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:16.713601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:16.731775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f1909-4a2a-4875-bfb1-0fa171688edf None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:16.772664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b6473-cf0e-44b4-ab0e-a3e20460cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:16.773342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b6473-cf0e-44b4-ab0e-a3e20460cdda None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:16.773617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b6473-cf0e-44b4-ab0e-a3e20460cdda None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:16.773888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b6473-cf0e-44b4-ab0e-a3e20460cdda None admin] PATH_INFO: `/v3/projects/98517b32cdd846b8863aae632875ea20/users/8656ec8244ac4ec688951c2ad60e3904/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:16.775745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b6473-cf0e-44b4-ab0e-a3e20460cdda None admin] RBAC: Authorizing `identity:create_grant(project_id=98517b32cdd846b8863aae632875ea20, user_id=8656ec8244ac4ec688951c2ad60e3904, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:16.777200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b6473-cf0e-44b4-ab0e-a3e20460cdda None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:16.855439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 604/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:17:16 2022] PUT /identity/v3/projects/98517b32cdd846b8863aae632875ea20/users/8656ec8244ac4ec688951c2ad60e3904/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:17:18.979940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dc9b0-0ef6-4a96-9c61-818e2a68d095 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:18.980486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dc9b0-0ef6-4a96-9c61-818e2a68d095 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:18.980486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dc9b0-0ef6-4a96-9c61-818e2a68d095 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:18.981774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 589/1193] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:18.987644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:18.988083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:18.988249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:19.006146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:19.007317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:19.070612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 605/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:17:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:17:19.341257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dc9b0-0ef6-4a96-9c61-818e2a68d095 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:19.417711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:19.439447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dc9b0-0ef6-4a96-9c61-818e2a68d095 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:19.471869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:19.472468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:19.472744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:19.473035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:19.483549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:17:19.488254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:17:19.489446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:17:19.490335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 590/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:17:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:17:19.495310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:19.562351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:19.592770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64757ca3-1359-40bd-bd9e-1a08c6292ae8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:19.626092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:19.626881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:19.627334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:19.627334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:19.627916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:19.628214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:19.629606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:19.637305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 606/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:17:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:19.643806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:19.740878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f55b8-252d-4dba-9dc4-c09f19908dbb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:19.773384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:19.774003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:19.774283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:19.774556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] PATH_INFO: `/v3/users/8656ec8244ac4ec688951c2ad60e3904` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:19.775660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] RBAC: Authorizing `identity:get_user(user_id=8656ec8244ac4ec688951c2ad60e3904)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:19.777110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:19.778648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 591/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Tue Oct 4 10:17:19 2022] GET /identity/v3/users/8656ec8244ac4ec688951c2ad60e3904 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:17:19.784363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:19.879771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24369-6544-4cc1-a3d0-ec4d2fc160c6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:19.907181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:19.912110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:19.912695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:19.913000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:19.913271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:19.978058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:19.979492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:20.072208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 607/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:17:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:17:20.078435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:20.082261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:20.175136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3eee2-f1b4-4f0a-b0e7-9e33d7d157b6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:20.207384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a212e1e-ae50-4230-a5ed-27712a584a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:20.207993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a212e1e-ae50-4230-a5ed-27712a584a4b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:20.208268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a212e1e-ae50-4230-a5ed-27712a584a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:20.208534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a212e1e-ae50-4230-a5ed-27712a584a4b None admin] PATH_INFO: `/v3/projects/98517b32cdd846b8863aae632875ea20` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:20.209647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a212e1e-ae50-4230-a5ed-27712a584a4b None admin] RBAC: Authorizing `identity:get_project(project_id=98517b32cdd846b8863aae632875ea20)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:20.211090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a212e1e-ae50-4230-a5ed-27712a584a4b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:20.212678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 592/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Tue Oct 4 10:17:20 2022] GET /identity/v3/projects/98517b32cdd846b8863aae632875ea20 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:20.218664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:20.247651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:20.312182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf52d2c-ef79-40da-abce-260de1e59aa1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:20.345346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:20.345933 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:20.346203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:20.346472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:20.347173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=98517b32cdd846b8863aae632875ea20, user.id=8656ec8244ac4ec688951c2ad60e3904) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:20.347473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:20.348904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e652e-aec7-4016-abc5-a0536dbd1e42 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:20.354315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 608/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:17:20 2022] GET /identity/v3/role_assignments?user.id=8656ec8244ac4ec688951c2ad60e3904&scope.project.id=98517b32cdd846b8863aae632875ea20&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:17:30.203245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=64 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:30.351840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=52 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:39.264625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4cac-e0f7-4e38-89bb-16edeebf1c7b None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:39.265249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4cac-e0f7-4e38-89bb-16edeebf1c7b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:39.265249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4cac-e0f7-4e38-89bb-16edeebf1c7b None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:39.266557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 593/1201] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:39.272290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:39.272779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:39.272908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:39.290644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:39.293356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:39.357192 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 609/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:18:39.634260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4cac-e0f7-4e38-89bb-16edeebf1c7b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:39.734630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4cac-e0f7-4e38-89bb-16edeebf1c7b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:39.767766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:39.768381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:39.768657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:39.768960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:39.781322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:39.782776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:39.796958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:39.799407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:39.800669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:39.800669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:39.800669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:39.800669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 594/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:39.805021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:39.900639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e0ff9-0e03-4de5-95f2-2957d9d2e63c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:39.929147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:39.934742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:39.935344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:39.935619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:39.935887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:39.936513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:39.936811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:39.953353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56b274-fbab-42ab-af65-cc26fc46c305 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:39.953353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 610/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:18:39.955721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:40.008554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:40.055970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaabf44-6646-4182-bfd3-41cbf9d8497b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:40.089193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6df117-04af-4566-ac49-dc0ae6650191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:40.089790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6df117-04af-4566-ac49-dc0ae6650191 None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:40.090065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6df117-04af-4566-ac49-dc0ae6650191 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:40.090333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6df117-04af-4566-ac49-dc0ae6650191 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:40.090975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6df117-04af-4566-ac49-dc0ae6650191 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:40.092380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6df117-04af-4566-ac49-dc0ae6650191 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:40.249376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 595/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Tue Oct 4 10:18:39 2022] POST /identity/v3/users => generated 251 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:18:42.422686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79246-9d86-436f-b42f-3ba221289d35 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:42.423259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79246-9d86-436f-b42f-3ba221289d35 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:42.423259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b79246-9d86-436f-b42f-3ba221289d35 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:42.424587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 611/1206] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:42.430246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:42.430679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:42.430806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:42.450045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:42.451230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:42.516972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 596/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:18:42.784614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b79246-9d86-436f-b42f-3ba221289d35 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:42.885925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b79246-9d86-436f-b42f-3ba221289d35 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:42.917829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:42.918441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:42.918716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:42.918986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:42.929235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:18:42.933929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:42.935063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:42.935947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 612/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:42.941894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:42.981763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:43.038481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9604e-01fc-4d3b-9950-562fd08d1b07 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:43.073215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:43.073820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:43.074357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:43.074631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:43.075290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:43.075591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:43.077075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:43.084709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 597/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:43.093799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:43.142285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:43.192000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4b313-5995-44a6-a27e-58de14461052 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:43.228204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:43.228886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:43.229172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:43.229443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:43.240362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:43.241894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:43.256013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:43.269640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:43.270997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:43.270997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:43.270997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:43.270997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 613/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:43.271314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:43.360389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36aee90-7753-42f2-82b1-b1cca55c0f58 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:43.392482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:43.393099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:43.393374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:43.393642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:43.394262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:43.394556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:43.396304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:43.404499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:43.409242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 598/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:43.415676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:43.515517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df871b-e028-43a9-823c-228ce6f820a0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:43.517047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:43.549951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:43.550575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:43.550860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:43.552414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:43.560065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:43.561556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:18:43.573850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:43.575218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:43.576242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 614/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:43.581108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:43.684077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3881a1-0f7b-431d-b8be-0a772425b248 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:43.719704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:43.720309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:43.720595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:43.720893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:43.721616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:43.721923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:43.723396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:43.741508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 599/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Oct 4 10:18:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:43.747322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:43.846403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab87a0-dcbd-471d-b97d-c59630798541 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:43.880221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:43.880777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:43.881096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:43.881374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:43.910246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:43.955471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:43.958018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:44.073420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 615/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:18:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:18:44.079494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:44.098536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:44.181185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e50fa5-b3f5-4a58-bced-483eb2ca70dc None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:44.215310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:44.215948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:44.216224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:44.216514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:44.227756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:44.233061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:44.244074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:44.245256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:44.246358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:44.246358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:44.246358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:44.246358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 600/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:44.251021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:44.324960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:44.366504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85895f0e-3165-436b-8895-41f1972fd87f None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:44.399639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:44.400267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:44.400551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:44.400823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:44.401513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:44.401932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:44.403328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:44.416112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 616/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:44.421857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:44.490177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:44.519152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0865b-d5c2-4795-859c-804997286647 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:44.551615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:44.552226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:44.552504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:44.552774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:44.559421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:44.561469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:18:44.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:44.573504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:44.574761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 601/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:44.579594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:44.683889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838b23e-140f-4462-b82e-54580bcd3d20 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:44.706480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:44.718529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:44.719139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:44.719421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:44.719698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:44.720384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:44.720688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:44.722162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03106d2f-7e19-4ce3-8fc4-6431ef797dee None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:44.733389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 617/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:18:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:44.739574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:44.840606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48326463-8b5d-4790-8951-9c233a4ba906 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:44.868844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:44.874561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:44.875162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:44.875441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:44.875718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:44.876479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=a172b6b42d6443029ce492937796e2fb) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:44.876786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:44.878254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59805be2-112f-46c1-990e-d01b36d8e471 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:44.883997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 602/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:18:44 2022] GET /identity/v3/role_assignments?user.id=a172b6b42d6443029ce492937796e2fb&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:47.078066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb4f06-b3be-45c7-a2f5-56de5b3d581a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.078608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb4f06-b3be-45c7-a2f5-56de5b3d581a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.078608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb4f06-b3be-45c7-a2f5-56de5b3d581a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.079947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 618/1220] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:47.085203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.085631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.085758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.104319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:47.105546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:47.174225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 603/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:18:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:18:47.445077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb4f06-b3be-45c7-a2f5-56de5b3d581a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:47.545921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb4f06-b3be-45c7-a2f5-56de5b3d581a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:47.578657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:47.579310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.582498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.582498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.590069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:18:47.594848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:47.596102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:47.597213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 619/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:18:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:47.602874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:47.701805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4970e2-f634-4076-9c93-35283b6189ba None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:47.726103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:47.736279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:47.736911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.737190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.737464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.738122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] RBAC: Adding query filter params (name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:47.738424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:47.739786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:47.747514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 604/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:18:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:47.753347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:47.796458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:47.851258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00ac50-46b5-4675-9de1-ba408c75dc1c None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:47.884973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:47.885631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.885909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.886211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.895822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:47.898348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:47.912244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:47.913544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:47.914470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:47.914470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:47.914470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:47.914470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 620/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:47.919945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.019540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe700e7-a1d2-4d83-b0a2-2dc26c7b0913 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:48.051819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:48.052413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.052689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.052982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.054219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:48.054494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:48.055844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:48.067383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 605/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:48.073984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.168207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:48.174753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e4ef-0e5a-4e84-8da4-5c480d12ad02 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:48.209516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:48.210657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.210936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.211207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.218913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:48.220530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:18:48.233048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:48.234050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:48.235291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 621/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:48.240055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.286333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:48.337988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febe0a8-17f5-4176-b839-5a9ba9cfdbaf None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:48.371840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:48.372434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.372715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.373010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.373715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:48.374017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:48.375425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:48.392674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 606/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Oct 4 10:18:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:48.398764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.498407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18176d3d-ac64-4ab4-9e7a-1f94a92e92a8 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:48.531040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:48.531665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.531946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.532230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.542923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:48.544320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:48.558670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:48.561473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:48.562417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:48.562417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:48.562417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:48.562417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 622/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:48.566001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.667431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9478d0b-a0bd-4c98-a2f6-cabb7eb0d868 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:48.669691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:48.701033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:48.701633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.701914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.702187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.702818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:48.703122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:48.704492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:48.715722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 607/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:48.721610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.731660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:48.821149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dada05-6b5b-444d-b32b-0fe4beaf2ca1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:48.854606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:48.855255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.856231 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.856512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.863334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:48.864786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:18:48.876709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:48.877889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:48.879321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 623/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:18:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:48.884372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:48.981902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d452b9-e35e-42f5-8066-d743a5e7044a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:49.014059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:49.014650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:49.014927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:49.015195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:49.015864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:49.016162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:49.017596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6f072-782b-41da-869a-48d9c084459a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:49.029983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 608/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:18:49.036140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:49.102090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:49.136272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fec452-16e7-4f2a-8dc7-85cc979ab704 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:49.169529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387f45-9674-4e21-b68b-b1f6f248728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:49.170171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387f45-9674-4e21-b68b-b1f6f248728a None admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:49.170445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387f45-9674-4e21-b68b-b1f6f248728a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:49.170714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387f45-9674-4e21-b68b-b1f6f248728a None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/a172b6b42d6443029ce492937796e2fb/roles/e8d669e07c8644589b6ac5b5b667fa9e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:49.185100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387f45-9674-4e21-b68b-b1f6f248728a None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=a172b6b42d6443029ce492937796e2fb, role_id=e8d669e07c8644589b6ac5b5b667fa9e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:49.186529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387f45-9674-4e21-b68b-b1f6f248728a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:49.289222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 624/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:18:49 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/a172b6b42d6443029ce492937796e2fb/roles/e8d669e07c8644589b6ac5b5b667fa9e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:18:51.448591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef079c-3a22-4bd4-b901-f5a2fbbcaaf8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:51.449447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef079c-3a22-4bd4-b901-f5a2fbbcaaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:51.449447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef079c-3a22-4bd4-b901-f5a2fbbcaaf8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:51.451327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 609/1233] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:51.456599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:51.457041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:51.457199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:51.474975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:51.476740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:51.542173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 625/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:18:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:18:51.820548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef079c-3a22-4bd4-b901-f5a2fbbcaaf8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:51.846147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:51.921185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef079c-3a22-4bd4-b901-f5a2fbbcaaf8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:51.953492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:51.954115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:51.954392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:51.954658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:51.964982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:18:51.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:51.975765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:51.976651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 610/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Tue Oct 4 10:18:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:18:51.977654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:52.022883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:52.077092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51f7c7-8cf9-4b59-8a90-9733d963fc67 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:52.110273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:52.110961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:52.111236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:52.111504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:52.112156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] RBAC: Adding query filter params (name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:52.112452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:52.113829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:52.121649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 626/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Tue Oct 4 10:18:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:52.127731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:52.214713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:52.230679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb805033-2cab-4e64-aa4c-12454e3a5fba None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:52.263769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:52.264392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:52.264671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:52.264968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:52.274805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:52.284558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:52.290779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:52.291886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:52.292798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:52.292798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:52.292798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:52.292798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 611/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:52.297631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:52.394411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4b52b-e165-4c98-8cff-0afa94a51979 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:52.402815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:52.426491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:52.427079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:52.427356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:52.427627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:52.428301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:52.429139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:52.430532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:52.441507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 627/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:52.447508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:52.545605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecd786-d141-48d7-b39a-f93c570da176 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:52.577563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:52.578179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:52.579354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:52.579354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:52.581331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:52.586650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:52.588072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:18:52.598681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:52.599863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:52.601782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 612/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:52.606410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:52.708240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467506b2-528f-44ae-b6e8-6cbac1b61b21 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:52.741903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:52.742495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:52.742768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:52.743079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:52.743788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:52.744087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:52.745523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:52.761861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 628/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Oct 4 10:18:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:52.767798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:52.871200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662a59a-82ec-4e5c-92f7-fe4f61750219 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:52.903989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:52.904591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:52.904879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:52.905168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:52.965268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:52.971503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:52.972974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.070529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 613/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:18:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:18:53.076701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:53.104012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:53.173009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10906-e72f-497b-9974-f03386bace2a None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:53.206934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:53.207575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.207849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.208120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.218427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:53.219828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:53.234183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:53.235269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:53.236415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:53.236415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:53.236415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:53.236415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 629/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:53.241696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:53.335711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:53.341355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41b9b-7c72-4e5b-98e2-f52db107f4c5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:53.374601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:53.375207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.375493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.375768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.376408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:53.376710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:53.378107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.390753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 614/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:53.396526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:53.474883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:53.496432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b711-a92d-4305-9703-b0c5fbc35924 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:53.529162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:53.530428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.530428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.530428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.536700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:53.538210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:18:53.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:53.550096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:53.551386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 630/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:53.556479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:53.655630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c349d-78d7-4bac-b756-f058e9f67b8a None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:53.689095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:53.689691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.689968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.690237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.690912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:53.691213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:53.692654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6799b02-a6e9-496c-a08e-cd4171405f00 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.704552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 615/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:53.711087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:53.807930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1f81a-c25e-4a10-9c9c-6bbd6005b14b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:53.840931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:53.841521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.841804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.842074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.842793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] RBAC: Adding query filter params (role.id=e8d669e07c8644589b6ac5b5b667fa9e, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=a172b6b42d6443029ce492937796e2fb) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:53.843099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:53.844524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce13f9-ef81-4cc4-99d4-e8465e622cca None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.850492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 631/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:18:53 2022] GET /identity/v3/role_assignments?user.id=a172b6b42d6443029ce492937796e2fb&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=e8d669e07c8644589b6ac5b5b667fa9e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:18:56.062139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de159b-5f17-474c-90de-dd2a7cc26e32 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:56.062702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de159b-5f17-474c-90de-dd2a7cc26e32 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:56.062702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de159b-5f17-474c-90de-dd2a7cc26e32 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:56.064209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 616/1247] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:56.070651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:56.075213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:56.075213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:56.077845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:56.090862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:56.092059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:56.157064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 632/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:18:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:18:56.426987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de159b-5f17-474c-90de-dd2a7cc26e32 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:56.446300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:56.526966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de159b-5f17-474c-90de-dd2a7cc26e32 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:56.561004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:56.561625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:56.561903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:56.562173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:56.572596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:18:56.577681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:56.578867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:56.579783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 617/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:56.585123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:56.682535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f154a7d-7012-4d1f-bac7-7081a384f7a5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:56.717171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:56.723735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:56.724625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:56.725613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:56.726031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:56.727017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:56.727509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:56.729703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:56.742235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 633/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:18:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:56.762667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:56.826414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:56.848587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b07a5d-99e0-429e-8f73-984ae9c6a095 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:56.882260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:56.882886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:56.883181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:56.883443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:56.893176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:56.894625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:56.910576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:56.911760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:56.912697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:56.912697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:56.912697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:56.912697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 618/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:56.918714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:57.017547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23c028-2584-4648-82ea-96aa5138e465 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:57.052811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:57.054455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:57.054610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:57.055014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:57.055957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:57.056411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:57.058576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:57.076321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 634/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:57.082449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:57.180216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422303ca-6ce9-4132-aaed-e4748fd1b650 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:57.205685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:57.213585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:57.214214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:57.214494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:57.214766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:57.222085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:57.223512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:18:57.234637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:57.235802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:18:57.237230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 619/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:57.242358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:57.304053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:57.347772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52ee63-882e-4470-b802-ad989d1d5b19 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:57.380994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:57.381614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:57.381900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:57.382170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:57.382888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:57.383191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:57.384627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:57.405036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 635/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Oct 4 10:18:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:57.407321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:57.508665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c26a-29fa-48c3-9207-8b2dc48c4da6 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:57.541468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:57.542061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:57.542335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:57.542603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:57.596451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:57.609020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:57.610479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:57.708710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 620/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:18:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:18:57.715131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:57.810647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b465a6-2c1b-461c-85ce-a55605fd0042 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:57.823156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:57.843702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:57.844320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:57.844592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:57.844875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:57.854922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:57.856299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:18:57.870048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:57.875552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:18:57.876740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:18:57.876740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:18:57.876740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:57.876740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 636/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:57.877059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:57.962058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:57.973336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33536f69-6c51-45d9-b6d4-cb3c4eeb327c None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:58.005917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:58.006647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:58.006879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:58.007147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:58.007780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:58.008076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:58.009454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:58.020106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 621/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:58.026594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:58.128292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d8259-4cb3-445b-a406-b1056ba198ab None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:58.165747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:58.166379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:58.166665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:58.166940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:58.173438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:58.174901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:18:58.186196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:18:58.193180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:18:58.194851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 637/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:18:58.195464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:58.293505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acf2c1-9bc1-4eae-94b5-285e770642c8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:58.326532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:58.327129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:58.327536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:58.327792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:58.328461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:58.328799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:58.330248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb69ff6-35d2-4843-bc14-56b54d31ca70 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:58.342756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 622/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:18:58.349018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:58.374064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:58.446522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c25a1-1c05-481a-9bbd-849143185c2b None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:58.485220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:58.485839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:58.486328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:58.486596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:58.487322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=a172b6b42d6443029ce492937796e2fb) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:18:58.487627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:58.489186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a567a0de-6588-4a3d-be24-60252ae5605b None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:58.495663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 638/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:18:58 2022] GET /identity/v3/role_assignments?user.id=a172b6b42d6443029ce492937796e2fb&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:00.669135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211eafe7-b3ee-470f-b639-dcb849666e04 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:00.669668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211eafe7-b3ee-470f-b639-dcb849666e04 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:00.669668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211eafe7-b3ee-470f-b639-dcb849666e04 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:00.671003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 623/1261] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:00.676139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:00.676569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:00.676691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:00.695836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:00.705091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:00.761667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 639/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:19:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:19:01.031678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211eafe7-b3ee-470f-b639-dcb849666e04 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:01.043248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:01.129716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211eafe7-b3ee-470f-b639-dcb849666e04 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:01.163791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:01.164419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:01.164694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:01.164991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:01.174769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:19:01.179539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:01.181386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:01.183203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 624/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:01.187404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:01.207642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:01.292219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4eda5-5af8-4557-b73c-e12969082676 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:01.332983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:01.333620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:01.333903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:01.334177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:01.335963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:01.336268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:01.337690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:01.351473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 640/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:19:01.357349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:01.431223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:01.464968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054183d-faf8-4e25-b828-2aaac718c48b None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:01.500810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:01.501463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:01.501739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:01.502008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:01.512327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:01.513786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:19:01.528333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:01.529389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:19:01.530432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:19:01.530432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:01.530432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:01.530432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 625/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:01.535237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:01.618187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:01.637228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149fcd5-9d98-4358-a83d-7716f5209453 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:01.670520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:01.671118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:01.671397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:01.671668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:01.672293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:01.672592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:01.673974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:01.685519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 641/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:01.691216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:01.791068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac38fdd-53c3-4661-821c-6e2d59052fee None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:01.811463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:01.825707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:01.826334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:01.826613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:01.826882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:01.834465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:01.838010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:19:01.846943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:01.848095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:01.849442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 626/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:19:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:01.854498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:01.953368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16189-d929-4960-a532-42d9f7508ac6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:01.986755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:01.987355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:01.987629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:01.987898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:01.988648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:01.988978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:01.990545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:01.999962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:02.007902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 642/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Oct 4 10:19:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:02.013736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:02.111558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377b6cc-c9cc-4daa-a7d6-dc6d45a3c3f4 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:02.144028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:02.144670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.144979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.145252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.156254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:02.159208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:19:02.172044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:02.173231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:19:02.174388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:19:02.174388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:02.174388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:02.174388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 627/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:02.179656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:02.276862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df4885-3cc3-494f-afa5-e6c1a7bdb0bd None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:02.309417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:02.310011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.310297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.310603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.311230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:02.311530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:02.312906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:02.323858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 643/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:02.330054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:02.361915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:02.427684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c5d1-1826-426a-af1d-bb05f1242862 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:02.460373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:02.461051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.461332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.461605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.468044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:02.469513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:19:02.481712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:02.484179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:02.485355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 628/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:19:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:02.489385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:02.546505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:02.587511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39108387-48d4-416f-9cc8-361806018a83 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:02.620749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:02.621388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.621670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.621947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.622756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:02.623064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:02.624503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83adb5-a585-4c65-8bdd-fec93d86664c None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:02.636979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 644/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:19:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:02.643150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:02.733592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:02.742266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99db96-875c-43c6-8ea4-eb5a06d07a83 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:02.775400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1015f4-ce58-4c75-bb4e-54db4062a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:02.776044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1015f4-ce58-4c75-bb4e-54db4062a10b None admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.776318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1015f4-ce58-4c75-bb4e-54db4062a10b None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.776585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1015f4-ce58-4c75-bb4e-54db4062a10b None admin] PATH_INFO: `/v3/projects/7383c7c14750435da1af9814d3b13f73/users/a172b6b42d6443029ce492937796e2fb/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.778456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1015f4-ce58-4c75-bb4e-54db4062a10b None admin] RBAC: Authorizing `identity:create_grant(project_id=7383c7c14750435da1af9814d3b13f73, user_id=a172b6b42d6443029ce492937796e2fb, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:02.779910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1015f4-ce58-4c75-bb4e-54db4062a10b None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:02.906244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 629/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Tue Oct 4 10:19:02 2022] PUT /identity/v3/projects/7383c7c14750435da1af9814d3b13f73/users/a172b6b42d6443029ce492937796e2fb/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:19:05.027281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887e5e-67af-4988-916e-a89376b4e0eb None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:05.036961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887e5e-67af-4988-916e-a89376b4e0eb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:05.036961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887e5e-67af-4988-916e-a89376b4e0eb None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:05.036961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 645/1274] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:05.037262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:05.037262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:05.037262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:05.054510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:05.055697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:05.120970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 630/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:19:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:19:05.391628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887e5e-67af-4988-916e-a89376b4e0eb None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:05.491826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887e5e-67af-4988-916e-a89376b4e0eb None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:05.512567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:05.530474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:05.531156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:05.531424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:05.531692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:05.542620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:19:05.547314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:05.548425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:05.549385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 646/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Tue Oct 4 10:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:05.554616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:05.637027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:05.654973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe66d21-a1f4-43ea-80b1-f3971d4417e8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:05.686924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:05.687532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:05.687813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:05.688087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:05.688749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:05.689080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:05.690450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:05.698769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 631/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Tue Oct 4 10:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:05.704923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:05.804910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a26f09-4573-4998-a37b-5aca379b69f9 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:05.839814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:05.840439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:05.840722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:05.841046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:05.850752 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:05.852192 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:19:05.867149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:05.869413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:19:05.870372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:19:05.870372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:05.870372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:05.870372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 647/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:05.874108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:05.973417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5d6c8-cad6-4db7-89d4-d8b65b7a61c5 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:06.006900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:06.007514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:06.007791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:06.008060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:06.008693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:06.009021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:06.010453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:06.022718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 632/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:06.028518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:06.059582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:06.127854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c3ad6-b579-4c9e-bdcd-7283d41b6210 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:06.160382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:06.161037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:06.161375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:06.161597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:06.168323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:06.169808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:19:06.182331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:06.183291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:06.184386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 648/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:19:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:19:06.189337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:06.194288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:06.285526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b51cad9-92ee-4151-a4b0-adad29c6226f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:06.318305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:06.319482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:06.319762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:06.320035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:06.320755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:06.321088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:06.322510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:06.339136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 633/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Tue Oct 4 10:19:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:06.344977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:06.434578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:06.443625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ec3b-fc6c-472c-8a1a-ed3f577b8cd1 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:06.477834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:06.478424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:06.478700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:06.478970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:06.545268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:06.546635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:06.634624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:06.644721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 649/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Tue Oct 4 10:19:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:19:06.651303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:06.711025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:06.748161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b632f-ce36-4822-9e94-42dbd9096e4f None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:06.782555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:06.783170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:06.783444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:06.783710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:06.793692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:06.795117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:19:06.809860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:06.810960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:19:06.812356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:19:06.812356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:19:06.812356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:06.812356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 634/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:06.816760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:06.918675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533748c8-87c3-478b-96eb-ef62ab4ca3b6 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:06.951903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:06.952513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:06.952783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:06.953080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:06.953738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:06.954037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:06.955388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:06.967074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 650/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Tue Oct 4 10:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:19:06.973037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:07.074073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aef29-2038-4f21-a101-f138ac3d60de None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:07.087875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:07.109444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:07.110060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:07.110335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:07.110606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:07.117605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:07.119039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:19:07.129828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:07.135113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: ERROR keystone.server.flask.application Oct 04 10:19:07.140904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 635/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Tue Oct 4 10:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:07.141531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:07.191975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:07.240470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d1a3-faf6-4a7b-90dd-68e0b00efabb None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:07.274901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:07.275561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:07.275841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:07.276246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:07.277132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:07.277511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:07.279675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6789-c46b-4bf7-96ca-aa21e4eb3602 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:07.292336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 651/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Tue Oct 4 10:19:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:07.298621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:07.400307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd599dc-5bbd-4a00-88f1-89fee036f9bb None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:07.432744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:07.433376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:07.433658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:07.433933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:07.434648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] RBAC: Adding query filter params (role.id=9211b6839ec4401ca9aada37827b6b8b, scope.project.id=7383c7c14750435da1af9814d3b13f73, user.id=a172b6b42d6443029ce492937796e2fb) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:07.434955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:07.436388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287b8a3-800c-410e-add7-68a4ff0c0f8e None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:07.441972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 636/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Tue Oct 4 10:19:07 2022] GET /identity/v3/role_assignments?user.id=a172b6b42d6443029ce492937796e2fb&scope.project.id=7383c7c14750435da1af9814d3b13f73&role.id=9211b6839ec4401ca9aada37827b6b8b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:09.600497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eee72-fbdf-4f0c-9dcd-0e6415c6c773 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:09.601110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eee72-fbdf-4f0c-9dcd-0e6415c6c773 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:09.601110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eee72-fbdf-4f0c-9dcd-0e6415c6c773 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:09.602385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 652/1288] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:09.607560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:09.608024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:09.619178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:09.627490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:09.628676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:09.658036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:09.694505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 637/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:19:09.965801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eee72-fbdf-4f0c-9dcd-0e6415c6c773 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:10.066113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eee72-fbdf-4f0c-9dcd-0e6415c6c773 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:10.077655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:10.100446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:10.101094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:10.101375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:10.101646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:10.102220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:10.103587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:10.111926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:10.116955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:10.118048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 653/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:19:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:19:10.119850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:10.215686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767827e-160c-4134-92f1-dbb753e53e5a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:10.249428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:10.249965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:10.250243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:10.250513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:10.251135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] RBAC: Adding query filter params (name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:10.251435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:10.252796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995a7b5-8858-4651-9dfa-0d4b16647cbe None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:10.257574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 638/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Oct 4 10:19:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:10.263560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:10.362173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418c23f-0d74-4d19-9b2a-1a5e7425621e None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:10.398865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:10.399469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:10.399766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:10.400028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] PATH_INFO: `/v3/services` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:10.400655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:10.400996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:10.402356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3d2fd-d7a9-47b9-8a31-2d03f8b8ce0f None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:10.406620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 654/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Oct 4 10:19:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:12.570426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e944-a42a-4b16-a297-4ee6e56c22c5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:12.570990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e944-a42a-4b16-a297-4ee6e56c22c5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:12.570990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e944-a42a-4b16-a297-4ee6e56c22c5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:12.572621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 639/1293] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:12.578616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2787c-9274-4d65-b164-92b9c9795b5e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:12.579139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2787c-9274-4d65-b164-92b9c9795b5e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:12.579255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd2787c-9274-4d65-b164-92b9c9795b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:12.598800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-edd2787c-9274-4d65-b164-92b9c9795b5e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:12.600548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-edd2787c-9274-4d65-b164-92b9c9795b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:12.637415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:12.680273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 655/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:19:12.957775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e944-a42a-4b16-a297-4ee6e56c22c5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:12.991705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:13.062209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e944-a42a-4b16-a297-4ee6e56c22c5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:13.096211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6583a9a-7366-47ba-baa8-272c4a121ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:13.096816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6583a9a-7366-47ba-baa8-272c4a121ffa None admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:13.097159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6583a9a-7366-47ba-baa8-272c4a121ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:13.097436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6583a9a-7366-47ba-baa8-272c4a121ffa None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:13.098149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6583a9a-7366-47ba-baa8-272c4a121ffa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:13.099518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6583a9a-7366-47ba-baa8-272c4a121ffa None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:13.186163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 640/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Tue Oct 4 10:19:12 2022] POST /identity/v3/services => generated 246 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:19:15.416880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde982b-246a-4d7d-98e4-283a352002ce None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:15.417480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde982b-246a-4d7d-98e4-283a352002ce None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:15.417480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde982b-246a-4d7d-98e4-283a352002ce None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:15.418787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 656/1296] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:15.424079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:15.424510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:15.424628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:15.442707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:15.444392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:15.516068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 641/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:19:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:19:15.807552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde982b-246a-4d7d-98e4-283a352002ce None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:15.906995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde982b-246a-4d7d-98e4-283a352002ce None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:15.942444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:15.943073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:15.943351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:15.944201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:15.944925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:15.946294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:15.955523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:15.959749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:15.960912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 657/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:19:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:15.963100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:15.980996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:16.058647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b0cc3-39b1-4e66-b650-9c5a50b066de None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:16.092007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:16.092639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:16.092987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:16.093260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:16.093882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:16.094187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:16.095546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:16.100646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 642/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Oct 4 10:19:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:16.106667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:16.138623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:16.203719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722405f2-1ac6-4f74-9b7f-a5ce36181455 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:16.236833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:16.237459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:16.237739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:16.238016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:16.238706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] RBAC: Adding query filter params (interface=public, service_id=99e66949d92e4bb587823902bcd84ca3, region_id=RegionOne) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:16.239011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:16.240430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2049e6-ffe7-43b9-b0c7-bca6fea93a97 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:16.245873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 658/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Tue Oct 4 10:19:16 2022] GET /identity/v3/endpoints?service_id=99e66949d92e4bb587823902bcd84ca3&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:19:16.251486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:16.333069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:16.351466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f957a-06b5-4a6c-9ba9-a205650face8 None admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:16.385800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb89327-6316-41b1-88b8-8805a198e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:16.386620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb89327-6316-41b1-88b8-8805a198e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:16.387304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb89327-6316-41b1-88b8-8805a198e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:16.387569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb89327-6316-41b1-88b8-8805a198e0d8 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:16.388164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb89327-6316-41b1-88b8-8805a198e0d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:16.389549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb89327-6316-41b1-88b8-8805a198e0d8 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:16.396285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 643/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Tue Oct 4 10:19:16 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:19:18.566133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9608d-e932-4f03-b826-101eed3e526c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:18.566723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9608d-e932-4f03-b826-101eed3e526c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:18.566723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9608d-e932-4f03-b826-101eed3e526c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:18.568007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 659/1302] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:18.573788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:18.574201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:18.574616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:18.593550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:18.605133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:18.660674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 644/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:19:18 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:19:18.930705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9608d-e932-4f03-b826-101eed3e526c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:19.023239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:19.029630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9608d-e932-4f03-b826-101eed3e526c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:19.062624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:19.063233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:19.063509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:19.063780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:19.064446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:19.065823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:19:19.075379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:19:19.079599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application with Lock( Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: ERROR keystone.server.flask.application Oct 04 10:19:19.080661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 660/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Tue Oct 4 10:19:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:19:19.082672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:19.156456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:19.177907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d4151-e1a9-4f0f-ab70-a1f67111a8bd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:19.211637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:19.212161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:19.212463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:19.212733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] PATH_INFO: `/v3/services` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:19.213422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:19:19.213724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:19.215078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d58f1f-2981-4d64-8d0c-62049a82cdd1 None admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:19.220167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 645/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Tue Oct 4 10:19:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:19:19.226323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:19.324738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289176-df9c-4a0d-976d-0b23124ed4e0 None admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:19.360208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe743d-d959-4c78-b9ae-3d749517b51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:19.360803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe743d-d959-4c78-b9ae-3d749517b51a None admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:19.361105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe743d-d959-4c78-b9ae-3d749517b51a None admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:19.361378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe743d-d959-4c78-b9ae-3d749517b51a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:19.362275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe743d-d959-4c78-b9ae-3d749517b51a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:19.363651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe743d-d959-4c78-b9ae-3d749517b51a None admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:19.466483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 661/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:19:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:19:29.225797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=30 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:29.461673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:49.522562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951658-8e05-4add-ae15-ae20fd9f7ac0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:49.523003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951658-8e05-4add-ae15-ae20fd9f7ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:49.523111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951658-8e05-4add-ae15-ae20fd9f7ac0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:49.525053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 646/1307] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:26:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:26:49.533963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f8c7-d967-42ea-b3ef-adb600d10885 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:49.534744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f8c7-d967-42ea-b3ef-adb600d10885 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:49.535225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f8c7-d967-42ea-b3ef-adb600d10885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:49.646163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0a67f8c7-d967-42ea-b3ef-adb600d10885 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:49.647687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0a67f8c7-d967-42ea-b3ef-adb600d10885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:49.789010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 662/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:50.091917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cac99-dd10-4995-bcfa-eaa98aa0749f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:50.092222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cac99-dd10-4995-bcfa-eaa98aa0749f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:50.092493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cac99-dd10-4995-bcfa-eaa98aa0749f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:50.094132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 647/1309] 10.222.0.42 () {54 vars in 964 bytes} [Tue Oct 4 10:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:26:50.100820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2606-626d-451b-aaad-67cf064bcd23 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:50.101373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2606-626d-451b-aaad-67cf064bcd23 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:50.101848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d2606-626d-451b-aaad-67cf064bcd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:50.212122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ec0d2606-626d-451b-aaad-67cf064bcd23 None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:50.213872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ec0d2606-626d-451b-aaad-67cf064bcd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:50.242204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:50.329588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 663/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Tue Oct 4 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:50.333026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cac99-dd10-4995-bcfa-eaa98aa0749f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:50.336396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-390cac99-dd10-4995-bcfa-eaa98aa0749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:50.472740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cac99-dd10-4995-bcfa-eaa98aa0749f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:50.554260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:50.555016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:50.555182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:50.555634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:50.557443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:50.626466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:50.689756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:50.691462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef3bbe-7c51-458d-a8db-31decd8cd243 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:50.714143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 648/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:53.251516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81baee6-636f-4cf8-979c-ad1b697c2e20 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:53.251983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81baee6-636f-4cf8-979c-ad1b697c2e20 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:53.252088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81baee6-636f-4cf8-979c-ad1b697c2e20 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:53.253472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 664/1312] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:26:53.258792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f90534-a204-47ab-a9cb-4b85980e749d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:53.259215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f90534-a204-47ab-a9cb-4b85980e749d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:53.259340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f90534-a204-47ab-a9cb-4b85980e749d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:53.341566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-38f90534-a204-47ab-a9cb-4b85980e749d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:53.342780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-38f90534-a204-47ab-a9cb-4b85980e749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:53.445877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 649/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:53.737349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81baee6-636f-4cf8-979c-ad1b697c2e20 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:53.738882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b81baee6-636f-4cf8-979c-ad1b697c2e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:53.839648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81baee6-636f-4cf8-979c-ad1b697c2e20 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:53.908997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:53.911377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:53.911983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:53.912264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:53.912571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:53.914275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:54.021860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:54.023302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9b898-a8d5-4306-ab58-7f024fcaa301 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:54.042283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 665/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:56.713406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb06cc2-abf7-41d0-ac27-829da57b1198 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.713986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb06cc2-abf7-41d0-ac27-829da57b1198 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.713986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb06cc2-abf7-41d0-ac27-829da57b1198 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.715276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 650/1315] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:26:56.721168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76f67-2820-407e-82ca-7b52404e9df6 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.721589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76f67-2820-407e-82ca-7b52404e9df6 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.721756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76f67-2820-407e-82ca-7b52404e9df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.786271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-60a76f67-2820-407e-82ca-7b52404e9df6 None None] MFA Rules not processed for user `135d762e2a5b413aa4e27f4cc35ec7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:56.787496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-60a76f67-2820-407e-82ca-7b52404e9df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:56.839732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:56.882343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 666/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:57.168443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb06cc2-abf7-41d0-ac27-829da57b1198 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.170038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6bb06cc2-abf7-41d0-ac27-829da57b1198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:57.180353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:57.273543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb06cc2-abf7-41d0-ac27-829da57b1198 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.347210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.347812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.348093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.348368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.350007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:57.413214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:57.459062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.460503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903152e-2ff3-4ad9-95d7-c722472ead89 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.480520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 651/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:00.456143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314bebd-3b93-4791-a6a8-434ccc42c651 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.456695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314bebd-3b93-4791-a6a8-434ccc42c651 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.456695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314bebd-3b93-4791-a6a8-434ccc42c651 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.458053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 667/1318] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:00.463232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799b24-3025-4f67-be71-f02bec8a70ce None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.463642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799b24-3025-4f67-be71-f02bec8a70ce None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.463774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799b24-3025-4f67-be71-f02bec8a70ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.536752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ea799b24-3025-4f67-be71-f02bec8a70ce None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:00.538028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ea799b24-3025-4f67-be71-f02bec8a70ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:00.635273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 652/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:00.951483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314bebd-3b93-4791-a6a8-434ccc42c651 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.953361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3314bebd-3b93-4791-a6a8-434ccc42c651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.056342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314bebd-3b93-4791-a6a8-434ccc42c651 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:01.128567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:01.129205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:01.129486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:01.129761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.131349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.138894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:01.244001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:01.245396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc3875-0ffb-43e7-934d-db0af33088d3 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:01.262697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 668/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:04.416142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8358d7-4e5d-44fe-9132-96c2f3317f83 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:04.416716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8358d7-4e5d-44fe-9132-96c2f3317f83 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:04.416716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8358d7-4e5d-44fe-9132-96c2f3317f83 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:04.418052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 653/1321] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:04.423711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93bdf1-b734-4c16-a1b6-2f13ed738050 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:04.424126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93bdf1-b734-4c16-a1b6-2f13ed738050 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:04.424259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be93bdf1-b734-4c16-a1b6-2f13ed738050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:04.490395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-be93bdf1-b734-4c16-a1b6-2f13ed738050 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:04.491623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-be93bdf1-b734-4c16-a1b6-2f13ed738050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:04.553266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:04.596003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 669/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:04.899798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59871d-7c02-4457-9216-afb3ee2de8e1 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:04.900142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59871d-7c02-4457-9216-afb3ee2de8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:04.900348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59871d-7c02-4457-9216-afb3ee2de8e1 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:04.901742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 654/1323] 10.222.0.42 () {54 vars in 964 bytes} [Tue Oct 4 10:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:04.908230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90f5ed-ce70-4612-858b-335c44a0a6ed None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:04.908514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90f5ed-ce70-4612-858b-335c44a0a6ed None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:04.908791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90f5ed-ce70-4612-858b-335c44a0a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:04.986006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2e90f5ed-ce70-4612-858b-335c44a0a6ed None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:04.987210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2e90f5ed-ce70-4612-858b-335c44a0a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:05.063519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 670/1324] 10.222.0.42 () {58 vars in 1061 bytes} [Tue Oct 4 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:05.072426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59871d-7c02-4457-9216-afb3ee2de8e1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:05.073988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ff59871d-7c02-4457-9216-afb3ee2de8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:05.094417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:05.177594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59871d-7c02-4457-9216-afb3ee2de8e1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:05.252926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:05.253535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:05.253817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:05.254093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:05.255693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:05.328192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:05.365465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:05.366925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9374-173a-44b1-bf2e-5032a7329699 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:05.385895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 655/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:07.904840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8372c0-407b-4dd0-aa99-fb0a2b93bd09 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:07.905466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8372c0-407b-4dd0-aa99-fb0a2b93bd09 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:07.905466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8372c0-407b-4dd0-aa99-fb0a2b93bd09 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:07.906968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 671/1326] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:07.912458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376729b-57c6-4e5c-8ed0-a3fe18236f42 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:07.912917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376729b-57c6-4e5c-8ed0-a3fe18236f42 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:07.913116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376729b-57c6-4e5c-8ed0-a3fe18236f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:07.981086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b376729b-57c6-4e5c-8ed0-a3fe18236f42 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:07.982344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b376729b-57c6-4e5c-8ed0-a3fe18236f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:08.082379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 656/1327] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:08.366057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8372c0-407b-4dd0-aa99-fb0a2b93bd09 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:08.367705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8a8372c0-407b-4dd0-aa99-fb0a2b93bd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:08.433774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:08.479104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8372c0-407b-4dd0-aa99-fb0a2b93bd09 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:08.557633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:08.558247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:08.558534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:08.558809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:08.560403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:08.671226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:08.674283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:08.675736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa9552-24ec-4e75-8e4d-8e870cfbaafb service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:08.694421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 672/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:10.961210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6566-6ab8-41d9-9a19-929a7ef1b8d8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:10.961748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6566-6ab8-41d9-9a19-929a7ef1b8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:10.961748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf6566-6ab8-41d9-9a19-929a7ef1b8d8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:10.962998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 657/1329] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:10.968366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efbafb-f171-4016-91d0-6c6748ac42fe None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:10.968761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efbafb-f171-4016-91d0-6c6748ac42fe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:10.968927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efbafb-f171-4016-91d0-6c6748ac42fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:11.039548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-68efbafb-f171-4016-91d0-6c6748ac42fe None None] MFA Rules not processed for user `135d762e2a5b413aa4e27f4cc35ec7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:11.040780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-68efbafb-f171-4016-91d0-6c6748ac42fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:11.142942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 673/1330] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:11.428089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf6566-6ab8-41d9-9a19-929a7ef1b8d8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:11.429648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2edf6566-6ab8-41d9-9a19-929a7ef1b8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:11.463816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:11.541128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf6566-6ab8-41d9-9a19-929a7ef1b8d8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:11.631660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:11.632411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:11.632674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:11.632968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:11.634557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:11.717839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:11.751585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:11.753067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a5825-4b49-4718-ad22-e65e1c29e536 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:11.771698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 658/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:14.290840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6f01-d8e3-44d5-9e94-093d7ec3edc5 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.291385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6f01-d8e3-44d5-9e94-093d7ec3edc5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.291385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd6f01-d8e3-44d5-9e94-093d7ec3edc5 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.292743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 674/1332] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:14.299711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2c849-e382-428e-bd6c-7da2cfa783f3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.300178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2c849-e382-428e-bd6c-7da2cfa783f3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.300285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2c849-e382-428e-bd6c-7da2cfa783f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.364734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1aa2c849-e382-428e-bd6c-7da2cfa783f3 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:14.365971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1aa2c849-e382-428e-bd6c-7da2cfa783f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:14.463148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 659/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:14.757401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd6f01-d8e3-44d5-9e94-093d7ec3edc5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.758947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-75dd6f01-d8e3-44d5-9e94-093d7ec3edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:14.826389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:14.860545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd6f01-d8e3-44d5-9e94-093d7ec3edc5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.933603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.934130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.934402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.934668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.936241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:15.044466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:15.045936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3791937-04bd-4518-be74-f2b208ff265a service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:15.055894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:15.063882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 675/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:17.292167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ae051-d06c-4ed3-9025-3eb38aca59c6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:17.292702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ae051-d06c-4ed3-9025-3eb38aca59c6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:17.292702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ae051-d06c-4ed3-9025-3eb38aca59c6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:17.294078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 660/1335] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:17.299853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8641f3-efea-49f7-8261-6a8a51c47ba9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:17.300311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8641f3-efea-49f7-8261-6a8a51c47ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:17.300408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8641f3-efea-49f7-8261-6a8a51c47ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:17.365677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-fe8641f3-efea-49f7-8261-6a8a51c47ba9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:17.366870 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fe8641f3-efea-49f7-8261-6a8a51c47ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:17.578733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 676/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:17.863487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ae051-d06c-4ed3-9025-3eb38aca59c6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:17.865125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9a8ae051-d06c-4ed3-9025-3eb38aca59c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:17.907281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:17.970167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ae051-d06c-4ed3-9025-3eb38aca59c6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:18.042197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:18.042796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:18.043074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:18.043347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:18.044964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:18.126869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:18.152063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:18.153527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029d0e9-59ed-4f8d-a982-32689b33c6c3 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:18.172721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 661/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:21.008115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca55d6-409b-4a3b-860c-91200975d626 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:21.008656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca55d6-409b-4a3b-860c-91200975d626 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:21.008656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca55d6-409b-4a3b-860c-91200975d626 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:21.010007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 677/1338] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:21.015133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c0bc-1317-4bf1-92ed-d8d5e240b9f6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:21.015520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c0bc-1317-4bf1-92ed-d8d5e240b9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:21.015687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c0bc-1317-4bf1-92ed-d8d5e240b9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:21.080419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5640c0bc-1317-4bf1-92ed-d8d5e240b9f6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:21.081649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5640c0bc-1317-4bf1-92ed-d8d5e240b9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:21.175184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 662/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:21.458111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca55d6-409b-4a3b-860c-91200975d626 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:21.459788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e7ca55d6-409b-4a3b-860c-91200975d626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:21.536011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:21.566942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca55d6-409b-4a3b-860c-91200975d626 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:21.647773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:21.648393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:21.648674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:21.648989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:21.650684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:21.760126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:21.761608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d32aad-6866-4d06-8846-dc21b93ccda8 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:21.779285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 678/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:26.093749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de65e1-61ca-4f36-bf2f-05a9beaa032f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:26.094276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de65e1-61ca-4f36-bf2f-05a9beaa032f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:26.094276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de65e1-61ca-4f36-bf2f-05a9beaa032f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:26.095560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 663/1341] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:26.100945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dc4c0-15e8-4e35-99a0-e8116eac022b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:26.101396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dc4c0-15e8-4e35-99a0-e8116eac022b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:26.101513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dc4c0-15e8-4e35-99a0-e8116eac022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:26.106765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:26.169080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9f7dc4c0-15e8-4e35-99a0-e8116eac022b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:26.170289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9f7dc4c0-15e8-4e35-99a0-e8116eac022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:26.268451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 679/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:26.553521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de65e1-61ca-4f36-bf2f-05a9beaa032f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:26.555057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-02de65e1-61ca-4f36-bf2f-05a9beaa032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:26.605707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:26.675411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de65e1-61ca-4f36-bf2f-05a9beaa032f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:26.751557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:26.752263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:26.752543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:26.752817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:26.754443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:26.851342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:26.866998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:26.868440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97683971-642b-4020-af26-1daf2cefe22c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:26.887796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 664/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:30.425309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef399d-64db-433d-9632-c08e8dfc818c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:30.425860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef399d-64db-433d-9632-c08e8dfc818c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:30.425860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef399d-64db-433d-9632-c08e8dfc818c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:30.427104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 680/1344] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:30.432591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4657c36-f17c-435a-9f45-9e652d2fca46 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:30.433112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4657c36-f17c-435a-9f45-9e652d2fca46 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:30.433206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4657c36-f17c-435a-9f45-9e652d2fca46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:30.502699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e4657c36-f17c-435a-9f45-9e652d2fca46 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:30.503918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e4657c36-f17c-435a-9f45-9e652d2fca46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:30.614937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 665/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:30.903397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef399d-64db-433d-9632-c08e8dfc818c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:30.905068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f9ef399d-64db-433d-9632-c08e8dfc818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.014130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:31.034495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef399d-64db-433d-9632-c08e8dfc818c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.112263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.112888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.113177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.113445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.115183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.244367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:31.246002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c7bfb-7492-4225-b888-abe279757259 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:31.276593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 681/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:35.816432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43ff02-78ed-4ccb-96d2-995c02f51cb4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:35.817019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43ff02-78ed-4ccb-96d2-995c02f51cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:35.817019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43ff02-78ed-4ccb-96d2-995c02f51cb4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:35.818387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 666/1347] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:35.823748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ee345-5595-4175-9a4c-6d64ba3edfb3 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:35.824194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ee345-5595-4175-9a4c-6d64ba3edfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:35.824300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ee345-5595-4175-9a4c-6d64ba3edfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:35.840739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:35.893161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5d9ee345-5595-4175-9a4c-6d64ba3edfb3 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:35.894425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5d9ee345-5595-4175-9a4c-6d64ba3edfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:35.994980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 682/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:36.278816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43ff02-78ed-4ccb-96d2-995c02f51cb4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:36.281023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8b43ff02-78ed-4ccb-96d2-995c02f51cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:36.344722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:36.392004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43ff02-78ed-4ccb-96d2-995c02f51cb4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:36.463435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:36.464032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:36.464306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:36.464578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:36.466181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:36.571771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:36.575025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:36.576450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7110054-ebf6-4912-aba3-2cefb06e2a4c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:36.604443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 667/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:41.642942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc358df-7b1c-422c-9ecc-68dd5fee332a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:41.644471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc358df-7b1c-422c-9ecc-68dd5fee332a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:41.645372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc358df-7b1c-422c-9ecc-68dd5fee332a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:41.647347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 683/1350] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:27:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:41.652993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69a47c-9ebc-4597-8b7c-f78315ed20fb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:41.653431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69a47c-9ebc-4597-8b7c-f78315ed20fb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:41.653550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69a47c-9ebc-4597-8b7c-f78315ed20fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:41.806845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-da69a47c-9ebc-4597-8b7c-f78315ed20fb None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:41.808064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-da69a47c-9ebc-4597-8b7c-f78315ed20fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:41.941052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 668/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:27:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:45.993365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=59 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:46.159846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8c22-da80-4096-a17a-80260edccd05 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:46.160437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8c22-da80-4096-a17a-80260edccd05 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:46.160833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c8c22-da80-4096-a17a-80260edccd05 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:46.162572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 684/1352] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:46.167692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ce7b5-677d-43dd-8ffd-c74867fd03ec None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:46.168189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ce7b5-677d-43dd-8ffd-c74867fd03ec None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:46.168460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ce7b5-677d-43dd-8ffd-c74867fd03ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:46.227584 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:46.251511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-487ce7b5-677d-43dd-8ffd-c74867fd03ec None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:46.252914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-487ce7b5-677d-43dd-8ffd-c74867fd03ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:46.366545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 669/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:47.691331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c522a-5729-428e-b008-0d905fbb8c85 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:47.691875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c522a-5729-428e-b008-0d905fbb8c85 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:47.691875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c522a-5729-428e-b008-0d905fbb8c85 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:47.693498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 685/1354] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:47.698573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc83b8b-8ef6-4a19-b90d-295a6abdc742 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:47.699017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc83b8b-8ef6-4a19-b90d-295a6abdc742 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:47.699244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc83b8b-8ef6-4a19-b90d-295a6abdc742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:47.783211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8fc83b8b-8ef6-4a19-b90d-295a6abdc742 None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:47.784437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8fc83b8b-8ef6-4a19-b90d-295a6abdc742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:47.830468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:47.906290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 670/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:51.643025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dd56c-8ee0-4191-8466-0242ddc60abc None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:51.644434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dd56c-8ee0-4191-8466-0242ddc60abc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:51.644969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5dd56c-8ee0-4191-8466-0242ddc60abc None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:51.646983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 686/1356] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:51.654883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e517e1-c523-4636-8705-a9654daeb699 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:51.655829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e517e1-c523-4636-8705-a9654daeb699 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:51.656171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e517e1-c523-4636-8705-a9654daeb699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:51.741189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b3e517e1-c523-4636-8705-a9654daeb699 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:51.742675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b3e517e1-c523-4636-8705-a9654daeb699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:51.851302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 671/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:52.165070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305739de-fd1e-4c16-8a5e-b2bb48b42118 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:52.165546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305739de-fd1e-4c16-8a5e-b2bb48b42118 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:52.165976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305739de-fd1e-4c16-8a5e-b2bb48b42118 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:52.167721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 687/1358] 10.222.0.42 () {54 vars in 961 bytes} [Tue Oct 4 10:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:52.178638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bfef2-75b7-436b-b1ab-5cb9b5cabcbc None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:52.178927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bfef2-75b7-436b-b1ab-5cb9b5cabcbc None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:52.179207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bfef2-75b7-436b-b1ab-5cb9b5cabcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:52.207797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:52.272780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a61bfef2-75b7-436b-b1ab-5cb9b5cabcbc None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:52.274047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a61bfef2-75b7-436b-b1ab-5cb9b5cabcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:52.394112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 672/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Oct 4 10:27:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:52.401725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305739de-fd1e-4c16-8a5e-b2bb48b42118 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:52.403238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-305739de-fd1e-4c16-8a5e-b2bb48b42118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:52.582862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305739de-fd1e-4c16-8a5e-b2bb48b42118 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:52.668962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:52.711359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:52.712047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:52.712305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:52.712996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:52.715389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:52.812795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:52.814775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de1ea8-8919-4df1-a11f-133bf4805b3e service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:52.836022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 688/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:55.226185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4e87f-c83e-4f5f-b3db-6e941758334d None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:55.226802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4e87f-c83e-4f5f-b3db-6e941758334d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:55.226802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4e87f-c83e-4f5f-b3db-6e941758334d None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:55.227984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 673/1361] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:55.233649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3c213-2311-4edf-912f-12931902d00d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:55.234094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3c213-2311-4edf-912f-12931902d00d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:55.234176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3c213-2311-4edf-912f-12931902d00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:55.300176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f4a3c213-2311-4edf-912f-12931902d00d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:55.301316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f4a3c213-2311-4edf-912f-12931902d00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:55.339670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:55.404434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 689/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:55.708471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4e87f-c83e-4f5f-b3db-6e941758334d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:55.710001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dfd4e87f-c83e-4f5f-b3db-6e941758334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:55.783732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:55.883427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4e87f-c83e-4f5f-b3db-6e941758334d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:56.026759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:56.032329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:56.032329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:56.032329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:56.034948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:56.060780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:56.132708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:56.134679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233e04b-9010-4190-92f8-ac2d95d026fb service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:56.155597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 674/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:58.795496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c149-c37f-4d7c-bdb9-a439cf82a46f None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:58.796030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c149-c37f-4d7c-bdb9-a439cf82a46f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:58.796333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c149-c37f-4d7c-bdb9-a439cf82a46f None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:58.797875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 690/1364] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:58.803385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d802424-0024-4ebc-9ec2-4e2c480b9921 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:58.803858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d802424-0024-4ebc-9ec2-4e2c480b9921 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:58.803959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d802424-0024-4ebc-9ec2-4e2c480b9921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:58.883292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1d802424-0024-4ebc-9ec2-4e2c480b9921 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:58.884418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1d802424-0024-4ebc-9ec2-4e2c480b9921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:58.994691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:59.001606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 675/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:59.289905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ea633-6234-4b99-a581-1931974e655d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:59.290346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ea633-6234-4b99-a581-1931974e655d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:59.290803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ea633-6234-4b99-a581-1931974e655d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:59.292403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 691/1366] 10.222.0.42 () {54 vars in 961 bytes} [Tue Oct 4 10:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:59.297925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405080a-d603-495f-83c6-cd1b2df391fc None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:59.298314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405080a-d603-495f-83c6-cd1b2df391fc None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:59.298589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405080a-d603-495f-83c6-cd1b2df391fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:59.380135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0405080a-d603-495f-83c6-cd1b2df391fc None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:59.381371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0405080a-d603-495f-83c6-cd1b2df391fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:59.506221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 676/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Oct 4 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:59.515256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ea633-6234-4b99-a581-1931974e655d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:59.516883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-664ea633-6234-4b99-a581-1931974e655d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:59.675550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ea633-6234-4b99-a581-1931974e655d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:59.703817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:59.801325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:59.801990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:59.802256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:59.802477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:59.804072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:59.905852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:59.907454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aba69c-0e36-4d82-9335-64837a3de73b service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:59.927010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 692/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:59.950233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525928bb-ff45-4302-93af-32ecc344b780 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:59.951400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525928bb-ff45-4302-93af-32ecc344b780 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:59.951400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525928bb-ff45-4302-93af-32ecc344b780 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:59.953751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 677/1369] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:27:59.959273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23656a-01ed-418c-9581-112ff7f8bd05 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:59.959806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23656a-01ed-418c-9581-112ff7f8bd05 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:59.961844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23656a-01ed-418c-9581-112ff7f8bd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:59.994838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:00.035733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-dc23656a-01ed-418c-9581-112ff7f8bd05 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:00.037045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-dc23656a-01ed-418c-9581-112ff7f8bd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:00.142300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 693/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:00.491854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd38d1-f6bb-4904-a01a-5470aa58211e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:00.492290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd38d1-f6bb-4904-a01a-5470aa58211e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:00.492568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd38d1-f6bb-4904-a01a-5470aa58211e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:00.494160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 678/1371] 10.222.0.42 () {54 vars in 963 bytes} [Tue Oct 4 10:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:00.499151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf5f73-fef5-4e2a-a318-7314cf273b5f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:00.499505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf5f73-fef5-4e2a-a318-7314cf273b5f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:00.499785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf5f73-fef5-4e2a-a318-7314cf273b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:00.582377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f1bf5f73-fef5-4e2a-a318-7314cf273b5f None None] MFA Rules not processed for user `55965a77a0fd41ce8007cea14d081007`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:00.583608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f1bf5f73-fef5-4e2a-a318-7314cf273b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:00.634977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:00.666070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 694/1372] 10.222.0.42 () {58 vars in 1060 bytes} [Tue Oct 4 10:28:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:28:00.674567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd38d1-f6bb-4904-a01a-5470aa58211e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:00.676090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0bdd38d1-f6bb-4904-a01a-5470aa58211e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:00.734274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:00.794423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd38d1-f6bb-4904-a01a-5470aa58211e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:00.881291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:00.881899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:00.882175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:00.882451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:00.884039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:00.996978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:01.012900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:01.015379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a63f4c-5734-4a61-8623-cb6e933bf9ed service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:01.042301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 679/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:02.416053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28ffb3-1bdd-4ea6-9b09-20587591dc3e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:02.416660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28ffb3-1bdd-4ea6-9b09-20587591dc3e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:02.416660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28ffb3-1bdd-4ea6-9b09-20587591dc3e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:02.417984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 695/1374] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:02.423427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2feac-c541-4c4c-936a-474626e1e39e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:02.423842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2feac-c541-4c4c-936a-474626e1e39e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:02.423987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2feac-c541-4c4c-936a-474626e1e39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:02.491906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-22a2feac-c541-4c4c-936a-474626e1e39e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:02.493198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-22a2feac-c541-4c4c-936a-474626e1e39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:02.591795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 680/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:02.901018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28ffb3-1bdd-4ea6-9b09-20587591dc3e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:02.902534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ba28ffb3-1bdd-4ea6-9b09-20587591dc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:03.060581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28ffb3-1bdd-4ea6-9b09-20587591dc3e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:03.113742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:03.177898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:03.178839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:03.178839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:03.179057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:03.180591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:03.276118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:03.277764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d77635-27aa-433c-8fec-17e2c5787ee3 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:03.297607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 696/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:05.698485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e9af-069c-4d8f-a439-0f755a926be5 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:05.699051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e9af-069c-4d8f-a439-0f755a926be5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:05.699051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826e9af-069c-4d8f-a439-0f755a926be5 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:05.701147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 681/1377] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:05.706625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6709ce9-f39c-422a-b971-fb82acb5df28 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:05.707036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6709ce9-f39c-422a-b971-fb82acb5df28 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:05.707204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6709ce9-f39c-422a-b971-fb82acb5df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:05.775466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f6709ce9-f39c-422a-b971-fb82acb5df28 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:05.776952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f6709ce9-f39c-422a-b971-fb82acb5df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:05.784993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:05.882622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 697/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:06.183541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826e9af-069c-4d8f-a439-0f755a926be5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:06.186102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c826e9af-069c-4d8f-a439-0f755a926be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:06.252199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:06.344113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826e9af-069c-4d8f-a439-0f755a926be5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:06.498042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:06.498621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:06.498975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:06.499323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:06.501028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:06.523356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:06.599078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:06.600817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f058ab-231e-4e01-acb6-e299091efb55 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:06.625995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 682/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:09.061904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a15bf-df7b-4b3d-91e5-17ff15d80ced None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:09.062643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a15bf-df7b-4b3d-91e5-17ff15d80ced None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:09.062643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a15bf-df7b-4b3d-91e5-17ff15d80ced None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:09.063637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 698/1380] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:09.069079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76684-8657-4c85-b0f6-8c3da2fcddd8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:09.069633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76684-8657-4c85-b0f6-8c3da2fcddd8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:09.069633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c76684-8657-4c85-b0f6-8c3da2fcddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:09.145007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-52c76684-8657-4c85-b0f6-8c3da2fcddd8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:09.146716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-52c76684-8657-4c85-b0f6-8c3da2fcddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:09.256610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:09.263979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 683/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:28:09.552496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a15bf-df7b-4b3d-91e5-17ff15d80ced None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:09.554043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c99a15bf-df7b-4b3d-91e5-17ff15d80ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:09.715358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:09.726412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a15bf-df7b-4b3d-91e5-17ff15d80ced None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:09.842636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:09.843315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:09.843586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:09.843854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:09.845450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:09.948238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:09.950100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfff92a-8ff1-4f0f-affb-8ea960bd34d7 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:09.961986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:09.970760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 699/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:12.807556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79ca92-4d34-4bbf-a189-7d21111c53ec None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:12.808116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79ca92-4d34-4bbf-a189-7d21111c53ec None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:12.808241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79ca92-4d34-4bbf-a189-7d21111c53ec None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:12.810179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 684/1383] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:12.816685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4343a60-e756-4e63-bcb7-fb524f09be28 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:12.817178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4343a60-e756-4e63-bcb7-fb524f09be28 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:12.817311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4343a60-e756-4e63-bcb7-fb524f09be28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:12.887373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d4343a60-e756-4e63-bcb7-fb524f09be28 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:12.889310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d4343a60-e756-4e63-bcb7-fb524f09be28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:13.004658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 700/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:13.296651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79ca92-4d34-4bbf-a189-7d21111c53ec None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:13.298199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1b79ca92-4d34-4bbf-a189-7d21111c53ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:13.462553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79ca92-4d34-4bbf-a189-7d21111c53ec None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:13.537211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:13.589010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:13.589654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:13.589945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:13.590234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:13.592576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:13.705489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:13.707239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e142e-9150-40b3-8646-cd8866e9ac30 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:13.725881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 685/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:16.000468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0502-29e4-4b99-9807-4aa48dad688d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:16.001045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0502-29e4-4b99-9807-4aa48dad688d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:16.001045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db0502-29e4-4b99-9807-4aa48dad688d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:16.002452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 701/1386] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:16.007816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4585f6-827b-40e5-b137-954ec2d764d8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:16.008232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4585f6-827b-40e5-b137-954ec2d764d8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:16.008360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4585f6-827b-40e5-b137-954ec2d764d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:16.076030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ee4585f6-827b-40e5-b137-954ec2d764d8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:16.077079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ee4585f6-827b-40e5-b137-954ec2d764d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:16.110838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:16.177497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 686/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:16.473448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db0502-29e4-4b99-9807-4aa48dad688d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:16.474966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e7db0502-29e4-4b99-9807-4aa48dad688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:16.568490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:16.649020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db0502-29e4-4b99-9807-4aa48dad688d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:16.771469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:16.772098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:16.772376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:16.772653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:16.774359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:16.817856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:16.878553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:16.880182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04252f3b-70fe-4968-9553-defd34f3818d service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:16.898571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 702/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:17.672446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cb347-cc5b-441a-846f-d9ac240db4b2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:17.673024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cb347-cc5b-441a-846f-d9ac240db4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:17.673024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cb347-cc5b-441a-846f-d9ac240db4b2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:17.674452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 687/1389] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:17.679716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440eae2-e284-4c0e-94f0-211bec99cbae None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:17.680203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440eae2-e284-4c0e-94f0-211bec99cbae None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:17.680351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440eae2-e284-4c0e-94f0-211bec99cbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:17.749630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f440eae2-e284-4c0e-94f0-211bec99cbae None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:17.750790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f440eae2-e284-4c0e-94f0-211bec99cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:17.849264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 703/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:18.162126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995fdb7-acf4-4947-b0ee-d936ca957b31 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.162401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995fdb7-acf4-4947-b0ee-d936ca957b31 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.162671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995fdb7-acf4-4947-b0ee-d936ca957b31 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.164040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 688/1391] 10.222.0.42 () {54 vars in 962 bytes} [Tue Oct 4 10:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:18.170518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23303fe-51a3-4483-a053-b5539e568685 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.170888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23303fe-51a3-4483-a053-b5539e568685 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.171239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23303fe-51a3-4483-a053-b5539e568685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.188493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:18.251077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c23303fe-51a3-4483-a053-b5539e568685 None None] MFA Rules not processed for user `28785166e8994ba18959790e0e5fd3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:18.252272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c23303fe-51a3-4483-a053-b5539e568685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.330085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 704/1392] 10.222.0.42 () {58 vars in 1059 bytes} [Tue Oct 4 10:28:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:18.339071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995fdb7-acf4-4947-b0ee-d936ca957b31 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.341194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8995fdb7-acf4-4947-b0ee-d936ca957b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.450029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995fdb7-acf4-4947-b0ee-d936ca957b31 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.511571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:18.527209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.527816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.528633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.528940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.530513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.646170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.647595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c8840-778b-441c-bb48-684ef0696665 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.666006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 689/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:19.227589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da544c-1cb4-4038-b424-516a4b1ad1c2 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.228007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da544c-1cb4-4038-b424-516a4b1ad1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.228151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da544c-1cb4-4038-b424-516a4b1ad1c2 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:19.229688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 705/1394] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:19.234949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.235384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.235481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:19.302654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:19.303859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:19.350758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:19.414037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 690/1395] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:28:19.591967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49739-d3b1-4d89-8a5e-2895c42327b2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.592245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49739-d3b1-4d89-8a5e-2895c42327b2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.592521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b49739-d3b1-4d89-8a5e-2895c42327b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:19.671175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b1b49739-d3b1-4d89-8a5e-2895c42327b2 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:19.671175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b1b49739-d3b1-4d89-8a5e-2895c42327b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:19.708947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:19.710173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:19.727669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:19.759787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 706/1396] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:19.810756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b848415-17ae-4e30-b57e-7dd9aa164d94 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.811036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b848415-17ae-4e30-b57e-7dd9aa164d94 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.811314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b848415-17ae-4e30-b57e-7dd9aa164d94 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:19.813156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 707/1397] 10.222.0.42 () {54 vars in 943 bytes} [Tue Oct 4 10:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:28:19.819323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5465-752c-482a-9545-422d278ed811 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.819786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5465-752c-482a-9545-422d278ed811 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.820062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5465-752c-482a-9545-422d278ed811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:19.842506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5d029-e7ac-460f-8ff1-dfe4a5cce4e1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:19.899680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:19.904255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c57e5465-752c-482a-9545-422d278ed811 None None] MFA Rules not processed for user `1e3ba7be5b48443697ad07dc4ffb04fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:19.905523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c57e5465-752c-482a-9545-422d278ed811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:19.958648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:19.959156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.959433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.959708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:19.961315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:20.015024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 708/1398] 10.222.0.42 () {58 vars in 1040 bytes} [Tue Oct 4 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:28:20.024475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5465-752c-482a-9545-422d278ed811 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:20.034414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5465-752c-482a-9545-422d278ed811 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:20.055850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:20.057545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905d2c3-72ae-4263-9811-c59ad8d9a8c1 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:20.060727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:20.077048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 691/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:20.139526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:20.140132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:20.140411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:20.140677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:20.142326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:20.232740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:20.234273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcc789-0a60-4b5a-8809-6351b8e1f946 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:20.242084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 709/1400] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:28:22.460676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3478f0-c9ee-4fff-b21e-a2b512f575b8 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:22.461307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3478f0-c9ee-4fff-b21e-a2b512f575b8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:22.461307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3478f0-c9ee-4fff-b21e-a2b512f575b8 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:22.462601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 692/1401] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:22.468232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06454fb-6f63-45e9-9d67-beff3572de5e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:22.468713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06454fb-6f63-45e9-9d67-beff3572de5e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:22.468805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06454fb-6f63-45e9-9d67-beff3572de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:22.548000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a06454fb-6f63-45e9-9d67-beff3572de5e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:22.549497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a06454fb-6f63-45e9-9d67-beff3572de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:22.556503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:22.658953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 710/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:22.956828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3478f0-c9ee-4fff-b21e-a2b512f575b8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:22.958371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ad3478f0-c9ee-4fff-b21e-a2b512f575b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:23.032684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:23.123833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3478f0-c9ee-4fff-b21e-a2b512f575b8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:23.256212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:23.256820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:23.257127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:23.257387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:23.258968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:23.288405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:23.363907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:23.366201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5d596-7d1d-4ad1-bf77-fbf48f0b35aa service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:23.384943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 693/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:28:25.758299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3ada7-4a18-4706-a2a6-8db4e58ff53a None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:25.758925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3ada7-4a18-4706-a2a6-8db4e58ff53a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:25.758925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3ada7-4a18-4706-a2a6-8db4e58ff53a None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:25.760198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 711/1404] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:25.766022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd90c0-ff68-492b-8de9-5829b94a351f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:25.766899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd90c0-ff68-492b-8de9-5829b94a351f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:25.767277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd90c0-ff68-492b-8de9-5829b94a351f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:25.854293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f8dd90c0-ff68-492b-8de9-5829b94a351f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:25.855585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f8dd90c0-ff68-492b-8de9-5829b94a351f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:25.968767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 694/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:26.264329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3ada7-4a18-4706-a2a6-8db4e58ff53a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:26.266030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bda3ada7-4a18-4706-a2a6-8db4e58ff53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:26.425002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:26.436258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3ada7-4a18-4706-a2a6-8db4e58ff53a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:26.564212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:26.564966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:26.565244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:26.565512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:26.567137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:26.675349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:26.677137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be626cbb-d0c9-4b4e-9f93-2c2cd7c6817e service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:26.688480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:26.696759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 712/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:29.092788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177eb5a-c800-4ad3-800c-5cabebeeac6a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:29.093444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177eb5a-c800-4ad3-800c-5cabebeeac6a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:29.093444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177eb5a-c800-4ad3-800c-5cabebeeac6a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:29.096031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 695/1407] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:29.101783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e40b4-878c-4230-84d0-2825035e7b12 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:29.102327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e40b4-878c-4230-84d0-2825035e7b12 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:29.102582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e40b4-878c-4230-84d0-2825035e7b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:29.177864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-812e40b4-878c-4230-84d0-2825035e7b12 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:29.179096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-812e40b4-878c-4230-84d0-2825035e7b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:29.287411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 713/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:29.586656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177eb5a-c800-4ad3-800c-5cabebeeac6a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:29.588187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4177eb5a-c800-4ad3-800c-5cabebeeac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:29.593044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:29.781965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177eb5a-c800-4ad3-800c-5cabebeeac6a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:29.862550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:29.907570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:29.908300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:29.908645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:29.909025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:29.910655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:30.010247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:30.011868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28fb51-bdea-499a-b2a7-653b10c8b9e4 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:30.035701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 696/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:39.285701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=67 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:40.028579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=61 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:44.390400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab115d-908b-42ad-9b60-c6f70c04327b None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.390998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab115d-908b-42ad-9b60-c6f70c04327b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.390998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab115d-908b-42ad-9b60-c6f70c04327b None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.392275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 714/1410] 10.222.0.42 () {54 vars in 918 bytes} [Tue Oct 4 10:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:44.397417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388b2d-d8c8-4085-a666-880e41b3f6e3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.397868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388b2d-d8c8-4085-a666-880e41b3f6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.397981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b388b2d-d8c8-4085-a666-880e41b3f6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.476845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7b388b2d-d8c8-4085-a666-880e41b3f6e3 None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:44.478081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7b388b2d-d8c8-4085-a666-880e41b3f6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:44.583951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 697/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Oct 4 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:44.599423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b392-bc71-44a9-b4ef-adcd106512d2 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.599779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b392-bc71-44a9-b4ef-adcd106512d2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.600064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b392-bc71-44a9-b4ef-adcd106512d2 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.601432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 715/1412] 10.222.0.42 () {54 vars in 918 bytes} [Tue Oct 4 10:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:44.605976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c9f2a-d3ac-4a3c-974e-87c31cdcac64 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.606257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c9f2a-d3ac-4a3c-974e-87c31cdcac64 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.606533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c9f2a-d3ac-4a3c-974e-87c31cdcac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.687759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:44.690758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-089c9f2a-d3ac-4a3c-974e-87c31cdcac64 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:44.691984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-089c9f2a-d3ac-4a3c-974e-87c31cdcac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:44.807073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 698/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Oct 4 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:47.419294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517fee-d073-413e-905e-594baf18284c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:47.419833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517fee-d073-413e-905e-594baf18284c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:47.419833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5517fee-d073-413e-905e-594baf18284c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:47.421099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:47.426421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f528c6-5f39-41c9-bf19-8052225b8b04 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:47.426850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f528c6-5f39-41c9-bf19-8052225b8b04 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:47.427003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f528c6-5f39-41c9-bf19-8052225b8b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:47.494690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-59f528c6-5f39-41c9-bf19-8052225b8b04 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:47.497013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-59f528c6-5f39-41c9-bf19-8052225b8b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:47.551957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:47.594519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:47.883854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5517fee-d073-413e-905e-594baf18284c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:47.885394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f5517fee-d073-413e-905e-594baf18284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:48.033631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5517fee-d073-413e-905e-594baf18284c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:48.105278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:48.146331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:48.146934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:48.147209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:48.147479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:48.149091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:48.238389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:48.240021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487e6bd-f559-4f6c-ba35-e24ea7d01cfe service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:48.258025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 717/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:49.604012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753d16-30e4-4781-882c-4628c895f987 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:49.604587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753d16-30e4-4781-882c-4628c895f987 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:49.604587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753d16-30e4-4781-882c-4628c895f987 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:49.606541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:49.613563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a67ed-ba07-4386-aa6e-d425e6265213 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:49.614116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a67ed-ba07-4386-aa6e-d425e6265213 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:49.616289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a67ed-ba07-4386-aa6e-d425e6265213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:49.703658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-da7a67ed-ba07-4386-aa6e-d425e6265213 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:49.705186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-da7a67ed-ba07-4386-aa6e-d425e6265213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:49.732683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:49.802695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:49.968746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753d16-30e4-4781-882c-4628c895f987 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:49.970213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a7753d16-30e4-4781-882c-4628c895f987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:50.113959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753d16-30e4-4781-882c-4628c895f987 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:50.153951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:50.226216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:50.226772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:50.227043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:50.227307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:50.228930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:50.317251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:50.318867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f16bbc-2f00-4233-a6f9-e4a963df0b79 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:50.336450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 701/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:51.464519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1395f-e6d8-48a4-b27a-812f73fc1ead None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:51.465130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1395f-e6d8-48a4-b27a-812f73fc1ead None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:51.465130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1395f-e6d8-48a4-b27a-812f73fc1ead None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:51.466492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:51.472651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7db62-a9e4-47d3-9e5c-cf65bea719e6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:51.473090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7db62-a9e4-47d3-9e5c-cf65bea719e6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:51.473228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7db62-a9e4-47d3-9e5c-cf65bea719e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:51.528704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:51.552022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f6f7db62-a9e4-47d3-9e5c-cf65bea719e6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:51.553723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f6f7db62-a9e4-47d3-9e5c-cf65bea719e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:51.664787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:51.825676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1395f-e6d8-48a4-b27a-812f73fc1ead None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:51.827126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e2e1395f-e6d8-48a4-b27a-812f73fc1ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:51.973396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1395f-e6d8-48a4-b27a-812f73fc1ead None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:51.985414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:52.087084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:52.087601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:52.087875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:52.088142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:52.090369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:52.177659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:52.179275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7add6-de3e-4e43-8bfe-c2f9d9df0f28 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:52.198031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 720/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:28:53.338220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1232-abf0-442f-9cc6-6ee25d5d0d82 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:53.338792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1232-abf0-442f-9cc6-6ee25d5d0d82 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:53.338792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff1232-abf0-442f-9cc6-6ee25d5d0d82 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:53.340822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:53.347098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa500ca1-eb09-4d02-903d-a428efe380c5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:53.347554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa500ca1-eb09-4d02-903d-a428efe380c5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:53.347654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa500ca1-eb09-4d02-903d-a428efe380c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:53.367144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:53.423922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-fa500ca1-eb09-4d02-903d-a428efe380c5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:53.425164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fa500ca1-eb09-4d02-903d-a428efe380c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:53.524617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:53.675479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1232-abf0-442f-9cc6-6ee25d5d0d82 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:53.677062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7fff1232-abf0-442f-9cc6-6ee25d5d0d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:53.791660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:53.828237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff1232-abf0-442f-9cc6-6ee25d5d0d82 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:53.938594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:53.939195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:53.939472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:53.939744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:53.941355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:54.023943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:54.043330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:54.045026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae9dd1-69a9-4990-8eb0-09c95685e3d9 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:54.066005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 704/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:55.179868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e0e9-7595-4c1e-9150-65a2e110ac7e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:55.180455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e0e9-7595-4c1e-9150-65a2e110ac7e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:55.180455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e0e9-7595-4c1e-9150-65a2e110ac7e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:55.181800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:55.188365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed37acf-40f0-45b2-9fbd-ab1b1ce13fa8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:55.188837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed37acf-40f0-45b2-9fbd-ab1b1ce13fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:55.188962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed37acf-40f0-45b2-9fbd-ab1b1ce13fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:55.263962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-eed37acf-40f0-45b2-9fbd-ab1b1ce13fa8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:55.265198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-eed37acf-40f0-45b2-9fbd-ab1b1ce13fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:55.360530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:55.510130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1e0e9-7595-4c1e-9150-65a2e110ac7e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:55.512175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4ee1e0e9-7595-4c1e-9150-65a2e110ac7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:55.597597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:55.659410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1e0e9-7595-4c1e-9150-65a2e110ac7e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:55.770343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:55.770878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:55.771150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:55.771416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:55.773029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:55.819778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:55.861453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:55.863043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14e2b6-7ebc-4a40-a71c-4368b4577286 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:55.880670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 723/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:57.047464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39bd5f-d647-43de-94a0-16184faab5a4 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:57.048029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39bd5f-d647-43de-94a0-16184faab5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:57.048029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39bd5f-d647-43de-94a0-16184faab5a4 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:57.049415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:57.055732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c26fc-0361-4d01-a39b-77802d446b96 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:57.056173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c26fc-0361-4d01-a39b-77802d446b96 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:57.056292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c26fc-0361-4d01-a39b-77802d446b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:57.133463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-eb1c26fc-0361-4d01-a39b-77802d446b96 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:57.134666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-eb1c26fc-0361-4d01-a39b-77802d446b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:57.231540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:57.379711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39bd5f-d647-43de-94a0-16184faab5a4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:57.381188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aa39bd5f-d647-43de-94a0-16184faab5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:57.430200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:57.527972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39bd5f-d647-43de-94a0-16184faab5a4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:57.644555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:57.645129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:57.645411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:57.645684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:57.647274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:57.654666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:57.738256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:57.740169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816ba75-272d-4fe3-b23a-7e69367aac7e service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:57.761630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 707/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:58.856368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce33c-6744-428c-926a-2c5e4993be8e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:58.856994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce33c-6744-428c-926a-2c5e4993be8e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:58.856994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce33c-6744-428c-926a-2c5e4993be8e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:58.858268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:28:58.864551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c65be-92ac-44ab-8c18-3ca0451f360e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:58.865038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c65be-92ac-44ab-8c18-3ca0451f360e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:58.865150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c65be-92ac-44ab-8c18-3ca0451f360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:58.944526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-431c65be-92ac-44ab-8c18-3ca0451f360e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:58.945777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-431c65be-92ac-44ab-8c18-3ca0451f360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:59.129457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:59.158960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:59.312123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dce33c-6744-428c-926a-2c5e4993be8e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:59.313678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a8dce33c-6744-428c-926a-2c5e4993be8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:59.331590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:59.456938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dce33c-6744-428c-926a-2c5e4993be8e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:59.543665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:59.569208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:59.569886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:59.570385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:59.570838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:59.572713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:59.665173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:59.666768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672083b-7b4d-4a13-be3f-5c10cd522dca service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:59.685683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 726/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:00.831092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9e8a-ba03-4914-a4c9-b82f176e97b0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:00.831669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9e8a-ba03-4914-a4c9-b82f176e97b0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:00.831669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9e8a-ba03-4914-a4c9-b82f176e97b0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:00.833033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:00.839513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26798a-d90f-4e32-b299-39e279cf3288 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:00.839963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26798a-d90f-4e32-b299-39e279cf3288 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:00.840076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26798a-d90f-4e32-b299-39e279cf3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:00.914558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ea26798a-d90f-4e32-b299-39e279cf3288 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:00.915691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ea26798a-d90f-4e32-b299-39e279cf3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:00.957238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:01.012005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:01.165561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9e8a-ba03-4914-a4c9-b82f176e97b0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:01.166999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e53f9e8a-ba03-4914-a4c9-b82f176e97b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:01.315182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9e8a-ba03-4914-a4c9-b82f176e97b0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:01.369044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:01.428287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:01.428815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:01.429123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:01.429393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:01.430995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:01.527484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:01.529120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401faf-4486-40f7-aa75-a63ee49cb263 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:01.547248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 710/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:02.712014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ca20b-b242-4b63-813d-8d870b877dbd None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:02.712587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ca20b-b242-4b63-813d-8d870b877dbd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:02.712587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ca20b-b242-4b63-813d-8d870b877dbd None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:02.714072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:02.720935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886b1c-cb3a-4983-8a74-63071a851f90 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:02.721370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886b1c-cb3a-4983-8a74-63071a851f90 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:02.721481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0886b1c-cb3a-4983-8a74-63071a851f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:02.798118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f0886b1c-cb3a-4983-8a74-63071a851f90 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:02.799336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f0886b1c-cb3a-4983-8a74-63071a851f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:02.801379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:02.914583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:03.063241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ca20b-b242-4b63-813d-8d870b877dbd None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:03.064749 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-125ca20b-b242-4b63-813d-8d870b877dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:03.212244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ca20b-b242-4b63-813d-8d870b877dbd None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:03.238877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:03.330498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:03.331102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:03.331379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:03.331650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:03.333272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:03.421620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:03.423229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85c6dc-5c52-45f8-92e2-06474b015413 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:03.441639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 729/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:05.561106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09a32-2fc8-4aef-9f08-9b75356db05a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:05.561679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09a32-2fc8-4aef-9f08-9b75356db05a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:05.561679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09a32-2fc8-4aef-9f08-9b75356db05a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:05.563019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:05.568413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf3c43-2f8e-4797-b154-76296b121e6c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:05.568852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf3c43-2f8e-4797-b154-76296b121e6c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:05.569019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf3c43-2f8e-4797-b154-76296b121e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:05.610057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:05.635910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-08bf3c43-2f8e-4797-b154-76296b121e6c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:05.637315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-08bf3c43-2f8e-4797-b154-76296b121e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:05.734846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:06.024302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09a32-2fc8-4aef-9f08-9b75356db05a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:06.025906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cfb09a32-2fc8-4aef-9f08-9b75356db05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:06.128553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09a32-2fc8-4aef-9f08-9b75356db05a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:06.160117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:06.200768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:06.201395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:06.201670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:06.201940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:06.203514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:06.310028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:06.311375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128ee8a-f934-405a-8f07-99fdf05a1352 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:06.329411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 713/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:08.677721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af06a4d-7d95-4d43-b6b9-5420d2405dbc None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:08.682892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af06a4d-7d95-4d43-b6b9-5420d2405dbc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:08.682892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af06a4d-7d95-4d43-b6b9-5420d2405dbc None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:08.682892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:08.685501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208598c-6566-4c81-82e9-7cd23d73bf63 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:08.685937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208598c-6566-4c81-82e9-7cd23d73bf63 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:08.686037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208598c-6566-4c81-82e9-7cd23d73bf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:08.754084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2208598c-6566-4c81-82e9-7cd23d73bf63 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:08.755330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2208598c-6566-4c81-82e9-7cd23d73bf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:08.768733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:08.851533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:09.146998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af06a4d-7d95-4d43-b6b9-5420d2405dbc None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:09.148537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9af06a4d-7d95-4d43-b6b9-5420d2405dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:09.259118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af06a4d-7d95-4d43-b6b9-5420d2405dbc None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:09.271888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:09.342603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:09.343206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:09.343483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:09.343754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:09.345379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:09.453809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:09.455421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21966a-84f4-4b34-a291-e19e7c58492e service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:09.474832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 732/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:11.987474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1eb2a-55d1-4554-8b74-dc0e1ec568c3 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:11.988009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1eb2a-55d1-4554-8b74-dc0e1ec568c3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:11.988009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1eb2a-55d1-4554-8b74-dc0e1ec568c3 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:11.989410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:11.995651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573681f-f369-41a6-9740-a087a99f3474 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:11.996144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573681f-f369-41a6-9740-a087a99f3474 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:11.996302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573681f-f369-41a6-9740-a087a99f3474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:12.043824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:12.063542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4573681f-f369-41a6-9740-a087a99f3474 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:12.064746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4573681f-f369-41a6-9740-a087a99f3474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:12.158106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:12.443796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1eb2a-55d1-4554-8b74-dc0e1ec568c3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:12.445361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-11c1eb2a-55d1-4554-8b74-dc0e1ec568c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:12.553224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1eb2a-55d1-4554-8b74-dc0e1ec568c3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:12.593889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:12.628416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:12.629066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:12.629350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:12.629622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:12.631227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:12.739928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:12.741370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b5fd1-c685-4833-9e1b-34f512431a06 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:12.759462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 716/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:15.018151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8513-828a-446f-8e15-f8135606505e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:15.018722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8513-828a-446f-8e15-f8135606505e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:15.018722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e8513-828a-446f-8e15-f8135606505e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:15.020020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:15.025295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cecff1-b634-4bcc-ba45-8080a66857ab None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:15.025746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cecff1-b634-4bcc-ba45-8080a66857ab None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:15.025888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cecff1-b634-4bcc-ba45-8080a66857ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:15.093626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-25cecff1-b634-4bcc-ba45-8080a66857ab None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:15.094824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-25cecff1-b634-4bcc-ba45-8080a66857ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:15.114616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:15.189876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:15.478346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8513-828a-446f-8e15-f8135606505e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:15.479961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-505e8513-828a-446f-8e15-f8135606505e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:15.584617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505e8513-828a-446f-8e15-f8135606505e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:15.604517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:15.660048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:15.660660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:15.660970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:15.661245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:15.662826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:15.781935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:15.783378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626f863-f698-4211-81d3-bf7406a676ff service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:15.801912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 735/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:18.047676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4141b-cec8-48a0-a1a6-609a94b87722 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:18.048247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4141b-cec8-48a0-a1a6-609a94b87722 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:18.048247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4141b-cec8-48a0-a1a6-609a94b87722 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:18.049612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:18.055005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f8f0-8bbe-4002-947d-bfc1c037d8fd None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:18.055398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f8f0-8bbe-4002-947d-bfc1c037d8fd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:18.055590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f8f0-8bbe-4002-947d-bfc1c037d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:18.122302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:18.125212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b0c8f8f0-8bbe-4002-947d-bfc1c037d8fd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:18.126430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0c8f8f0-8bbe-4002-947d-bfc1c037d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:18.222111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:18.513797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4141b-cec8-48a0-a1a6-609a94b87722 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:18.515344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8cf4141b-cec8-48a0-a1a6-609a94b87722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:18.623282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4141b-cec8-48a0-a1a6-609a94b87722 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:18.678961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:18.696167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:18.696767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:18.697071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:18.697342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:18.699490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:18.805292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:18.806718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fba7a-4481-4a53-8a00-be467ff9a071 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:18.824780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 719/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:21.059311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01e837-a1f8-49ed-8ce0-e500addbc89d None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:21.059884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01e837-a1f8-49ed-8ce0-e500addbc89d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:21.059884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01e837-a1f8-49ed-8ce0-e500addbc89d None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:21.061324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:21.066570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc236-08a0-4bb2-80c8-ef9db5ca2b0f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:21.066988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc236-08a0-4bb2-80c8-ef9db5ca2b0f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:21.067098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc236-08a0-4bb2-80c8-ef9db5ca2b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:21.132503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f83dc236-08a0-4bb2-80c8-ef9db5ca2b0f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:21.133730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f83dc236-08a0-4bb2-80c8-ef9db5ca2b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:21.164175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:21.232464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:21.517415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01e837-a1f8-49ed-8ce0-e500addbc89d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:21.519012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-da01e837-a1f8-49ed-8ce0-e500addbc89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:21.625924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01e837-a1f8-49ed-8ce0-e500addbc89d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:21.654894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:21.701503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:21.702225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:21.702573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:21.702885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:21.704731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:21.814098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:21.815458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcc39c-f34d-4ae6-ac9d-c9170d3b5b7a service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:21.833498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 738/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:24.026830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d40cf-dc23-46d4-8154-e43728b6d2ba None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:24.027382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d40cf-dc23-46d4-8154-e43728b6d2ba None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:24.027382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d40cf-dc23-46d4-8154-e43728b6d2ba None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:24.028705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:24.033935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525411e-b2ad-42e3-b6dc-b26c6b2f6157 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:24.034349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525411e-b2ad-42e3-b6dc-b26c6b2f6157 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:24.034483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525411e-b2ad-42e3-b6dc-b26c6b2f6157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:24.099328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3525411e-b2ad-42e3-b6dc-b26c6b2f6157 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:24.100524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3525411e-b2ad-42e3-b6dc-b26c6b2f6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:24.109125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:24.197053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:24.481240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d40cf-dc23-46d4-8154-e43728b6d2ba None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:24.482782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-128d40cf-dc23-46d4-8154-e43728b6d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:24.589218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d40cf-dc23-46d4-8154-e43728b6d2ba None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:24.646014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:24.663817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:24.664421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:24.664696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:24.665012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:24.667174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:24.778122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:24.779551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7915e-cf32-4ac6-89fc-b2a4986683b3 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:24.797075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 722/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:27.046222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f04fa-5eaf-4b04-a7dd-c9cdf6423487 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:27.046788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f04fa-5eaf-4b04-a7dd-c9cdf6423487 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:27.046788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f04fa-5eaf-4b04-a7dd-c9cdf6423487 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:27.048130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:27.053576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4c62-0e07-4346-9f1b-60bd46ecd306 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:27.053997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4c62-0e07-4346-9f1b-60bd46ecd306 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:27.054112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4c62-0e07-4346-9f1b-60bd46ecd306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:27.120200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-627e4c62-0e07-4346-9f1b-60bd46ecd306 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:27.121561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-627e4c62-0e07-4346-9f1b-60bd46ecd306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:27.158873 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:27.216959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:27.503977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f04fa-5eaf-4b04-a7dd-c9cdf6423487 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:27.505593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-425f04fa-5eaf-4b04-a7dd-c9cdf6423487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:27.609089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f04fa-5eaf-4b04-a7dd-c9cdf6423487 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:27.647627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:27.684647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:27.685295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:27.685573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:27.685847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:27.687467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:27.798091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:27.799524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433ccfe-1ac6-4817-af92-0ac7f7c3f2cc service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:27.817626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 741/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:29.977018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63433206-aae2-46bc-819f-dadf6557e6e6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:29.977596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63433206-aae2-46bc-819f-dadf6557e6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:29.977596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63433206-aae2-46bc-819f-dadf6557e6e6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:29.978749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:29.985116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88366a-3bde-4a65-9786-d2a4188c1032 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:29.985583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88366a-3bde-4a65-9786-d2a4188c1032 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:29.985693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88366a-3bde-4a65-9786-d2a4188c1032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:30.051085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2c88366a-3bde-4a65-9786-d2a4188c1032 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:30.052287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2c88366a-3bde-4a65-9786-d2a4188c1032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:30.070583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:30.148085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:30.430599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63433206-aae2-46bc-819f-dadf6557e6e6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:30.432173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-63433206-aae2-46bc-819f-dadf6557e6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:30.536695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63433206-aae2-46bc-819f-dadf6557e6e6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:30.604467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:30.611715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:30.612327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:30.612607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:30.612910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:30.614556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:30.725029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:30.726388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254f70b-0f22-4028-b8f6-0c6d738d1b48 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:30.744105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 725/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:32.931276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23212-730e-425f-9b5c-1b4efcf78060 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:32.931819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23212-730e-425f-9b5c-1b4efcf78060 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:32.931819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23212-730e-425f-9b5c-1b4efcf78060 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:32.933103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:32.938518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4514e52-b45f-4989-b2c9-11745bb403b4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:32.938923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4514e52-b45f-4989-b2c9-11745bb403b4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:32.939069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4514e52-b45f-4989-b2c9-11745bb403b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:33.005124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b4514e52-b45f-4989-b2c9-11745bb403b4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:33.006314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b4514e52-b45f-4989-b2c9-11745bb403b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:33.054013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:33.103349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:33.396993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23212-730e-425f-9b5c-1b4efcf78060 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:33.398537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f9a23212-730e-425f-9b5c-1b4efcf78060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:33.504370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23212-730e-425f-9b5c-1b4efcf78060 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:33.554219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:33.579874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:33.580482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:33.580760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:33.581080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:33.582662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:33.698607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:33.700609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3f529-c4f6-4152-994e-cc399276d361 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:33.719377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 744/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:43.101853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=22 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:43.717126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=66 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:54.093725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665b1c-6bf9-4229-885e-b580ae576cce None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:54.094304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665b1c-6bf9-4229-885e-b580ae576cce None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:54.094304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665b1c-6bf9-4229-885e-b580ae576cce None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:54.095665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 727/1471] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:54.100912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd0e03-b3ef-444a-a9df-e8be999a4674 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:54.101379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd0e03-b3ef-444a-a9df-e8be999a4674 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:54.101486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd0e03-b3ef-444a-a9df-e8be999a4674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:54.166702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e3cd0e03-b3ef-444a-a9df-e8be999a4674 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:54.168551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e3cd0e03-b3ef-444a-a9df-e8be999a4674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:54.264324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 745/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:29:54.567284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2daac-79ea-43f2-9bef-f80c0462d85a None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:54.567571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2daac-79ea-43f2-9bef-f80c0462d85a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:54.567849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2daac-79ea-43f2-9bef-f80c0462d85a None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:54.569207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 728/1473] 10.222.0.42 () {54 vars in 962 bytes} [Tue Oct 4 10:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:54.574280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837fd2e-0bfb-4c1a-a300-c5af143490e0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:54.574560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837fd2e-0bfb-4c1a-a300-c5af143490e0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:54.574833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837fd2e-0bfb-4c1a-a300-c5af143490e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:54.653333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9837fd2e-0bfb-4c1a-a300-c5af143490e0 None None] MFA Rules not processed for user `28785166e8994ba18959790e0e5fd3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:54.654549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9837fd2e-0bfb-4c1a-a300-c5af143490e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:54.675431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:54.732744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 746/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Tue Oct 4 10:29:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:54.740499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2daac-79ea-43f2-9bef-f80c0462d85a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:54.742054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f9f2daac-79ea-43f2-9bef-f80c0462d85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:54.847619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2daac-79ea-43f2-9bef-f80c0462d85a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:54.920184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:54.920781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:54.921085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:54.921351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:54.922919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:54.976549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:55.030929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:55.032361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5499467-0a1f-4fd7-80d1-8453b38e46f5 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:55.050545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 729/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:57.284709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3a12c-3ef8-4f3d-bb90-e45c578e02f5 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:57.285159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3a12c-3ef8-4f3d-bb90-e45c578e02f5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:57.285449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3a12c-3ef8-4f3d-bb90-e45c578e02f5 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:57.286626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 747/1476] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:29:57.292008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734324b-8376-4ae3-829c-3a78b42fc2fb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:57.292525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734324b-8376-4ae3-829c-3a78b42fc2fb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:57.292525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734324b-8376-4ae3-829c-3a78b42fc2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:57.357575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c734324b-8376-4ae3-829c-3a78b42fc2fb None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:57.358770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c734324b-8376-4ae3-829c-3a78b42fc2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:57.454040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 730/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:57.733561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3a12c-3ef8-4f3d-bb90-e45c578e02f5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:57.735153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e5e3a12c-3ef8-4f3d-bb90-e45c578e02f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:57.861820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3a12c-3ef8-4f3d-bb90-e45c578e02f5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:57.910690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:57.955068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:57.955939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:57.956275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:57.956927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:57.957577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:57.959732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:29:57.959732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:29:57.959732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:29:57.960888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590153c-6671-468a-b25c-dded0c990a86 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:57.980632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 748/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Tue Oct 4 10:29:57 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:30:00.169187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9247f6-fc6a-4bcb-aaf9-1a45a7b269f0 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:00.169803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9247f6-fc6a-4bcb-aaf9-1a45a7b269f0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:00.169803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9247f6-fc6a-4bcb-aaf9-1a45a7b269f0 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:00.171153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 731/1479] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:30:00.176575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be49af-b076-419e-9651-0299cf308544 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:00.177045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be49af-b076-419e-9651-0299cf308544 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:00.177159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be49af-b076-419e-9651-0299cf308544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:00.244249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-82be49af-b076-419e-9651-0299cf308544 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:00.246230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-82be49af-b076-419e-9651-0299cf308544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:00.343077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 749/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:00.642049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9247f6-fc6a-4bcb-aaf9-1a45a7b269f0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:00.643549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1d9247f6-fc6a-4bcb-aaf9-1a45a7b269f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:00.653772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:00.790772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9247f6-fc6a-4bcb-aaf9-1a45a7b269f0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:00.869455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:00.902437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:00.903043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:00.903325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:00.903600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:00.905221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:00.993822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:00.995658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933b597-d949-4cd4-ba01-54e5e7d70249 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:01.013819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 732/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:30:03.366739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bba012-17bd-47fa-b339-0c499e6cc257 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:03.367308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bba012-17bd-47fa-b339-0c499e6cc257 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:03.367308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bba012-17bd-47fa-b339-0c499e6cc257 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:03.368640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 750/1482] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:30:03.374737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af062-0f25-44d2-a458-874e325c0725 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:03.375152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af062-0f25-44d2-a458-874e325c0725 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:03.375284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af062-0f25-44d2-a458-874e325c0725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:03.440772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-022af062-0f25-44d2-a458-874e325c0725 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:03.442025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-022af062-0f25-44d2-a458-874e325c0725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:03.489062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:03.540284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 733/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:03.829329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bba012-17bd-47fa-b339-0c499e6cc257 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:03.831145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-29bba012-17bd-47fa-b339-0c499e6cc257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:03.855302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:03.935672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bba012-17bd-47fa-b339-0c499e6cc257 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:04.007945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:04.008547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:04.008824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:04.009121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:04.010906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:04.083703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:04.118396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:04.119880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdef90-30f0-4753-917f-139abe81c7e3 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:04.137514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 751/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:30:06.326654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bb769-3e68-4071-aed0-4d493a266653 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:06.327252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bb769-3e68-4071-aed0-4d493a266653 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:06.327252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bb769-3e68-4071-aed0-4d493a266653 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:06.329900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 734/1485] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:30:06.335216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45cdea-3146-4fff-b74e-bef949bd90a1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:06.335670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45cdea-3146-4fff-b74e-bef949bd90a1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:06.335767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45cdea-3146-4fff-b74e-bef949bd90a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:06.402807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ea45cdea-3146-4fff-b74e-bef949bd90a1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:06.404025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ea45cdea-3146-4fff-b74e-bef949bd90a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:06.499299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 752/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:06.784005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bb769-3e68-4071-aed0-4d493a266653 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:06.785602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a64bb769-3e68-4071-aed0-4d493a266653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:06.890548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bb769-3e68-4071-aed0-4d493a266653 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:06.965137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:06.966630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:06.967239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:06.967519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:06.967790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:06.969426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:07.078616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:07.080299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13321e6e-5aeb-458b-9c98-545a21e40f39 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:07.098466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 735/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:30:09.408843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef4c3a-b3e5-4560-a98c-ff135d7f886e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:09.409489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef4c3a-b3e5-4560-a98c-ff135d7f886e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:09.409489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef4c3a-b3e5-4560-a98c-ff135d7f886e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:09.410814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 753/1488] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:30:09.415961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313a521-04ee-40f3-aec9-cc0dc087424a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:09.416594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313a521-04ee-40f3-aec9-cc0dc087424a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:09.416594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313a521-04ee-40f3-aec9-cc0dc087424a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:09.483380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d313a521-04ee-40f3-aec9-cc0dc087424a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:09.484585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d313a521-04ee-40f3-aec9-cc0dc087424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:09.536419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:09.581400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 736/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:30:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:09.871913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef4c3a-b3e5-4560-a98c-ff135d7f886e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:09.873463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3fef4c3a-b3e5-4560-a98c-ff135d7f886e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:09.909219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:09.977162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef4c3a-b3e5-4560-a98c-ff135d7f886e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:10.052824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:10.053453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:10.053729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:10.053999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:10.055572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:10.133829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:10.162260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:10.163741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3ff81-cf9c-4ae1-b5bc-de7819c5d7d1 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:10.184804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 754/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:30:12.814282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136a51b-14a1-477d-a31f-fecc733eca4e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:12.814854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136a51b-14a1-477d-a31f-fecc733eca4e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:12.814854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136a51b-14a1-477d-a31f-fecc733eca4e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:12.816194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 737/1491] 10.222.0.42 () {54 vars in 927 bytes} [Tue Oct 4 10:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:30:12.821603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017057-4153-40b5-84f1-1d0afd219017 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:12.822016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017057-4153-40b5-84f1-1d0afd219017 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:12.822187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2017057-4153-40b5-84f1-1d0afd219017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:12.889860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c2017057-4153-40b5-84f1-1d0afd219017 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:12.890988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c2017057-4153-40b5-84f1-1d0afd219017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:12.985327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 755/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Tue Oct 4 10:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:13.276834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136a51b-14a1-477d-a31f-fecc733eca4e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:13.278405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9136a51b-14a1-477d-a31f-fecc733eca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:13.383091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136a51b-14a1-477d-a31f-fecc733eca4e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:13.458210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:13.466651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:13.466651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:13.466651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:13.466651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:13.470517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:13.571038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:13.572972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0af87-99ed-43cd-9335-660f965484ce service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:13.591717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 738/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:30:22.983893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=82 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:23.589604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=50 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:36.866006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:36.866580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:36.866580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:36.942669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:36.943883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:37.040237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 756/1494] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:37.048737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:37.049283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:37.049395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:37.127412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:37.128558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:37.226562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 739/1495] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:37.231255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:37.232462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:37.243614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6eb49-50ce-46be-a023-1a0ca1eaa130 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:37.303537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:37.337943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:37.338725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:37.339041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:37.339360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:37.340010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:31:37.340316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:37.341361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:37.341361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:31:37.341361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:31:37.342252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:37.353779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 757/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:31:37.362558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:37.371566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea091c-5c07-4ae8-803b-63d91b2a6c50 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:37.433423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:37.469526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:37.470587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:37.470860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:37.471134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:37.471757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:37.473521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:37.473521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:31:37.473521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:31:37.474708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:37.604738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 740/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:31:37 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:31:37.609350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:37.610932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:37.661664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b388d-9a1f-4b3c-92af-513f5c4409d4 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:37.772697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:37.773332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:37.773609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:37.773879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:37.774522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:37.775543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:37.775543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:31:37.775543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:31:37.777155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:37.913348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 758/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:31:37 2022] POST /identity/v3/users => generated 290 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:31:37.919247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:37.920766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:37.987127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd9e84-f34f-4990-8fbe-f3a9038e9bea admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:38.076587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:38.087538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:31:38.092215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:31:38.092921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:38.108710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 741/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:31:38.112852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:38.114471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:38.126797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eacacad-6727-43c5-9a23-a2710385027d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:38.152452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:38.230007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:38.230643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:38.230925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:38.231195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:38.231844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:38.232883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:38.232883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:31:38.232883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:31:38.233776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49cdbe-c5e8-478e-a9bb-d5ad510a85b4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:38.247840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 759/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:31:38.252186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:38.253798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:38.287129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50f95a-2388-4f85-9b1b-a78f88838650 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:38.387259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:38.387831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:38.388106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:38.388375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] PATH_INFO: `/v3/projects/5adf827522a24d1389cece5b444749f2/users/47f0a22d1a4f4f17a6cb4093e2552f40/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:38.411869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5adf827522a24d1389cece5b444749f2, user_id=47f0a22d1a4f4f17a6cb4093e2552f40, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:38.412939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:38.412939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:31:38.412939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:31:38.414911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f74801-4517-42d0-a82e-850c7a61391f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:38.524542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 742/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:31:38 2022] PUT /identity/v3/projects/5adf827522a24d1389cece5b444749f2/users/47f0a22d1a4f4f17a6cb4093e2552f40/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:31:38.543497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537c14d-b3a8-4b73-a55c-5c6c5e141dc4 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:38.543779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537c14d-b3a8-4b73-a55c-5c6c5e141dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:38.544055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0537c14d-b3a8-4b73-a55c-5c6c5e141dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:38.607516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0537c14d-b3a8-4b73-a55c-5c6c5e141dc4 None None] MFA Rules not processed for user `47f0a22d1a4f4f17a6cb4093e2552f40`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:38.609816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0537c14d-b3a8-4b73-a55c-5c6c5e141dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:38.705139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 760/1502] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:31:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:38.720320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936af6fb-3305-4eb2-a64d-258e5f91ce2c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:38.720596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936af6fb-3305-4eb2-a64d-258e5f91ce2c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:38.720916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936af6fb-3305-4eb2-a64d-258e5f91ce2c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:38.722258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 743/1503] 10.222.0.42 () {54 vars in 963 bytes} [Tue Oct 4 10:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:31:38.727753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dc35d-820b-47ce-99d7-2f4f5669e6ac None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:38.728033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dc35d-820b-47ce-99d7-2f4f5669e6ac None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:38.728307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dc35d-820b-47ce-99d7-2f4f5669e6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:38.761850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:38.813812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-195dc35d-820b-47ce-99d7-2f4f5669e6ac None None] MFA Rules not processed for user `55965a77a0fd41ce8007cea14d081007`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:38.815030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-195dc35d-820b-47ce-99d7-2f4f5669e6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:38.910963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 761/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Tue Oct 4 10:31:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:38.920310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936af6fb-3305-4eb2-a64d-258e5f91ce2c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:38.921858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-936af6fb-3305-4eb2-a64d-258e5f91ce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:38.981535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:39.049211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936af6fb-3305-4eb2-a64d-258e5f91ce2c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:39.132281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:39.133062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:39.133176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:39.133454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:39.135078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:39.286295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:39.287735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:39.291192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:39.306317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 744/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:39.352403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3865b2-eb64-4e5d-abe1-9c1a845f8b01 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:39.352832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3865b2-eb64-4e5d-abe1-9c1a845f8b01 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:39.352980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3865b2-eb64-4e5d-abe1-9c1a845f8b01 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:39.354361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 762/1506] 10.222.0.42 () {52 vars in 830 bytes} [Tue Oct 4 10:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:31:39.358078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:39.359284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:39.459778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a206100-f7d4-40a4-bf3b-628aced8433d service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:39.555545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:39.556067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:39.556336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:39.556603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] PATH_INFO: `/v3/users/47f0a22d1a4f4f17a6cb4093e2552f40` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:39.573609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f0a22d1a4f4f17a6cb4093e2552f40)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:39.575123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:39.575123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:31:39.575123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:31:39.576338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:39.588833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:39.809756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f0a22d1a4f4f17a6cb4093e2552f40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:39.810210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f0a22d1a4f4f17a6cb4093e2552f40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:39.824011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f0a22d1a4f4f17a6cb4093e2552f40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:39.824488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f0a22d1a4f4f17a6cb4093e2552f40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:39.901150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f0a22d1a4f4f17a6cb4093e2552f40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:39.933828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 745/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:31:39 2022] DELETE /identity/v3/users/47f0a22d1a4f4f17a6cb4093e2552f40 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:31:39.938346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:39.938877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:39.939183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:39.996977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:39.998189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:40.072157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:40.084256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 763/1508] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:40.095016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:40.096455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:40.196825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588e224-12ff-4e31-a645-01be1fa735e9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:40.253117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:40.271448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:40.272057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:40.272336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:40.272608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:40.274243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:40.389375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:40.390735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f282f0-91a0-45ff-8871-f3ce105ac2ef service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:40.409651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 746/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:40.928199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:40.929465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:40.943924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce7ff-8e6f-454f-a399-c4aad2ff26e0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:41.071135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:41.071803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:41.072079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:41.072348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] PATH_INFO: `/v3/projects/5adf827522a24d1389cece5b444749f2` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:41.086271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5adf827522a24d1389cece5b444749f2)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:41.087323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:31:41.087323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:31:41.087323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:31:41.088670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:41.226410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5adf827522a24d1389cece5b444749f2', 'request_id': 'req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:41.240643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5adf827522a24d1389cece5b444749f2', 'request_id': 'req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:41.324208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5adf827522a24d1389cece5b444749f2', 'request_id': 'req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:41.349810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] The token cache is being invalidate because project 5adf827522a24d1389cece5b444749f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:31:41.350078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:41.350511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a256b319-2f38-4aca-9bf5-ec5e8bd45c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:41.351970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 764/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:31:40 2022] DELETE /identity/v3/projects/5adf827522a24d1389cece5b444749f2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:31:50.407870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=62 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:51.346761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:08.110714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe913-9314-4a53-ba5c-e9628fad193c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:08.111321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe913-9314-4a53-ba5c-e9628fad193c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:08.111321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbe913-9314-4a53-ba5c-e9628fad193c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:08.112618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:08.117416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2c123-de5e-4817-9097-8f472f15578c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:08.117854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2c123-de5e-4817-9097-8f472f15578c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:08.117962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2c123-de5e-4817-9097-8f472f15578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:08.197904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-70e2c123-de5e-4817-9097-8f472f15578c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:08.199648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-70e2c123-de5e-4817-9097-8f472f15578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:08.301709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:08.455921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe913-9314-4a53-ba5c-e9628fad193c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:08.458299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4dcbe913-9314-4a53-ba5c-e9628fad193c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:08.575431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbe913-9314-4a53-ba5c-e9628fad193c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:08.580879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a084a9f-08b4-4eac-ad0d-56d21fa9960b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:08.581734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a084a9f-08b4-4eac-ad0d-56d21fa9960b service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:08.582020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a084a9f-08b4-4eac-ad0d-56d21fa9960b service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:08.582296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a084a9f-08b4-4eac-ad0d-56d21fa9960b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:08.590260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a084a9f-08b4-4eac-ad0d-56d21fa9960b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:08.591829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a084a9f-08b4-4eac-ad0d-56d21fa9960b service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:08.601124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 748/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:09.163562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7bb90-e5d0-4464-bb77-3a7bb406a71b None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:09.164135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7bb90-e5d0-4464-bb77-3a7bb406a71b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:09.164135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7bb90-e5d0-4464-bb77-3a7bb406a71b None None] PATH_INFO: `/v3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:09.165413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 766/1514] 10.222.0.45 () {54 vars in 925 bytes} [Tue Oct 4 10:33:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:33:09.171708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32113791-9085-4903-be95-27471589657f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:09.172146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32113791-9085-4903-be95-27471589657f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:09.172236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32113791-9085-4903-be95-27471589657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:09.234605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-32113791-9085-4903-be95-27471589657f None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:09.235799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-32113791-9085-4903-be95-27471589657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:09.301516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:09.306598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 749/1515] 10.222.0.45 () {58 vars in 1015 bytes} [Tue Oct 4 10:33:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:09.718990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fcb2e-9cf8-4ebd-be1b-c46f4f6a3f01 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:09.719255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fcb2e-9cf8-4ebd-be1b-c46f4f6a3f01 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:09.719534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fcb2e-9cf8-4ebd-be1b-c46f4f6a3f01 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:09.720992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 767/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:09.727173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52db33-4072-4e6c-8813-ad64cba88bbf None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:09.727463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52db33-4072-4e6c-8813-ad64cba88bbf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:09.727742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52db33-4072-4e6c-8813-ad64cba88bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:09.747266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bb52db33-4072-4e6c-8813-ad64cba88bbf None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:09.748469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bb52db33-4072-4e6c-8813-ad64cba88bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:09.756577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 750/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:09.910452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fcb2e-9cf8-4ebd-be1b-c46f4f6a3f01 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:09.924572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fcb2e-9cf8-4ebd-be1b-c46f4f6a3f01 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:09.929949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051af54-16c7-4458-8bf1-6025c2ee25e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:09.930536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051af54-16c7-4458-8bf1-6025c2ee25e2 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:09.930813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051af54-16c7-4458-8bf1-6025c2ee25e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:09.931085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051af54-16c7-4458-8bf1-6025c2ee25e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:09.939464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051af54-16c7-4458-8bf1-6025c2ee25e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:09.941065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051af54-16c7-4458-8bf1-6025c2ee25e2 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:09.951092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 768/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:18.054311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47778113-f70c-44b5-8dc7-fecaf754a921 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:18.054760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47778113-f70c-44b5-8dc7-fecaf754a921 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:18.054868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47778113-f70c-44b5-8dc7-fecaf754a921 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:18.056181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 751/1519] 10.222.0.42 () {54 vars in 961 bytes} [Tue Oct 4 10:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:18.061852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d082af-bdb6-4937-a854-50563111349d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:18.062311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d082af-bdb6-4937-a854-50563111349d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:18.062437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d082af-bdb6-4937-a854-50563111349d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:18.082122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-09d082af-bdb6-4937-a854-50563111349d None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:18.083330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-09d082af-bdb6-4937-a854-50563111349d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:18.092383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 769/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Oct 4 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:18.099832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47778113-f70c-44b5-8dc7-fecaf754a921 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:18.113658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47778113-f70c-44b5-8dc7-fecaf754a921 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:18.118525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f19a4-feb3-4232-93ce-d37400feac82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:18.119107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f19a4-feb3-4232-93ce-d37400feac82 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:18.119381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f19a4-feb3-4232-93ce-d37400feac82 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:18.119648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f19a4-feb3-4232-93ce-d37400feac82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:18.127767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f19a4-feb3-4232-93ce-d37400feac82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:18.129352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f19a4-feb3-4232-93ce-d37400feac82 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:18.138798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 752/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:21.455672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527526ae-634c-41ed-9d74-e04b496c520b None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:21.456225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527526ae-634c-41ed-9d74-e04b496c520b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:21.456225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527526ae-634c-41ed-9d74-e04b496c520b None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:21.457886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:21.464018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f315-a4fe-4be4-a014-5b0b43d10e5f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:21.464550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f315-a4fe-4be4-a014-5b0b43d10e5f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:21.464550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f315-a4fe-4be4-a014-5b0b43d10e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:21.484344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8837f315-a4fe-4be4-a014-5b0b43d10e5f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:21.485613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8837f315-a4fe-4be4-a014-5b0b43d10e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:21.493580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:21.656200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527526ae-634c-41ed-9d74-e04b496c520b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:21.671118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527526ae-634c-41ed-9d74-e04b496c520b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:21.676582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2542b1f-cbe6-4e4d-9707-783ecc4d5582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:21.677205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2542b1f-cbe6-4e4d-9707-783ecc4d5582 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:21.677590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2542b1f-cbe6-4e4d-9707-783ecc4d5582 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:21.677823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2542b1f-cbe6-4e4d-9707-783ecc4d5582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:21.685786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2542b1f-cbe6-4e4d-9707-783ecc4d5582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:21.689693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2542b1f-cbe6-4e4d-9707-783ecc4d5582 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:21.696698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 771/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:22.762789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c23755-4215-4b61-aab6-119e54186d26 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:22.763364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c23755-4215-4b61-aab6-119e54186d26 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:22.763364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c23755-4215-4b61-aab6-119e54186d26 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:22.764801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:22.770942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dde105-be83-42eb-8e3d-d5d1fb884649 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:22.771389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dde105-be83-42eb-8e3d-d5d1fb884649 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:22.771506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dde105-be83-42eb-8e3d-d5d1fb884649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:22.794065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-85dde105-be83-42eb-8e3d-d5d1fb884649 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:22.795855 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-85dde105-be83-42eb-8e3d-d5d1fb884649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:22.803856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:22.955404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c23755-4215-4b61-aab6-119e54186d26 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:22.970817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c23755-4215-4b61-aab6-119e54186d26 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:22.976196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49971-b8fd-4fca-b6b1-915769efa4da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:22.976793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49971-b8fd-4fca-b6b1-915769efa4da service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:22.977105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49971-b8fd-4fca-b6b1-915769efa4da service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:22.977379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49971-b8fd-4fca-b6b1-915769efa4da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:22.985277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd49971-b8fd-4fca-b6b1-915769efa4da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:22.986837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd49971-b8fd-4fca-b6b1-915769efa4da service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:22.995983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 755/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:27.159355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2f8dc-57f6-4093-8d1a-bea5d40c99dc None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:27.159939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2f8dc-57f6-4093-8d1a-bea5d40c99dc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:27.159939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2f8dc-57f6-4093-8d1a-bea5d40c99dc None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:27.161298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:27.166527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eadd9d-2b2e-415a-aa3e-9c497b0f5268 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:27.166951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eadd9d-2b2e-415a-aa3e-9c497b0f5268 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:27.167078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eadd9d-2b2e-415a-aa3e-9c497b0f5268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:27.186857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-30eadd9d-2b2e-415a-aa3e-9c497b0f5268 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:27.188060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-30eadd9d-2b2e-415a-aa3e-9c497b0f5268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:27.195878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:27.488481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2f8dc-57f6-4093-8d1a-bea5d40c99dc None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:27.490238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fea2f8dc-57f6-4093-8d1a-bea5d40c99dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:27.526037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:27.572668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2f8dc-57f6-4093-8d1a-bea5d40c99dc None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:27.576437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587b24c-86ad-44d6-8071-86aa3dd6e1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:27.579158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587b24c-86ad-44d6-8071-86aa3dd6e1cd service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:27.579158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587b24c-86ad-44d6-8071-86aa3dd6e1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:27.579158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587b24c-86ad-44d6-8071-86aa3dd6e1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:27.587078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587b24c-86ad-44d6-8071-86aa3dd6e1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:27.588530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587b24c-86ad-44d6-8071-86aa3dd6e1cd service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:27.598756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 774/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:29.738129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57ec36-2894-42f0-967b-d1c988dc6ae2 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:29.738706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57ec36-2894-42f0-967b-d1c988dc6ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:29.738706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57ec36-2894-42f0-967b-d1c988dc6ae2 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:29.740059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:29.745550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52f8f-9928-4993-8c36-87c4fa4783f8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:29.745978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52f8f-9928-4993-8c36-87c4fa4783f8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:29.746131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c52f8f-9928-4993-8c36-87c4fa4783f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:29.765803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-75c52f8f-9928-4993-8c36-87c4fa4783f8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:29.766998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-75c52f8f-9928-4993-8c36-87c4fa4783f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:29.774704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:30.061553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57ec36-2894-42f0-967b-d1c988dc6ae2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:30.072300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57ec36-2894-42f0-967b-d1c988dc6ae2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:30.075836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb03ef-f164-438a-b61c-e64dd5fe4a9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:30.077027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb03ef-f164-438a-b61c-e64dd5fe4a9a service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:30.077311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb03ef-f164-438a-b61c-e64dd5fe4a9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:30.077591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb03ef-f164-438a-b61c-e64dd5fe4a9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:30.086564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb03ef-f164-438a-b61c-e64dd5fe4a9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:30.087984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb03ef-f164-438a-b61c-e64dd5fe4a9a service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:30.097243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 758/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:33.846084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc2e30-8dc2-43b8-bdab-15d2be17edf5 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:33.846658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc2e30-8dc2-43b8-bdab-15d2be17edf5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:33.846658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc2e30-8dc2-43b8-bdab-15d2be17edf5 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:33.847980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:33.853249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c6737-3e4c-424b-b4bf-77d1d12ae17a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:33.853668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c6737-3e4c-424b-b4bf-77d1d12ae17a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:33.853826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c6737-3e4c-424b-b4bf-77d1d12ae17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:33.874179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c25c6737-3e4c-424b-b4bf-77d1d12ae17a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:33.875923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c25c6737-3e4c-424b-b4bf-77d1d12ae17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:33.883710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:34.188029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc2e30-8dc2-43b8-bdab-15d2be17edf5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:34.189776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-27fc2e30-8dc2-43b8-bdab-15d2be17edf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:34.267615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc2e30-8dc2-43b8-bdab-15d2be17edf5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:34.271240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399529f8-fe62-43c3-a626-b740b6a296e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:34.271843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399529f8-fe62-43c3-a626-b740b6a296e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:34.272125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399529f8-fe62-43c3-a626-b740b6a296e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:34.272402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399529f8-fe62-43c3-a626-b740b6a296e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:34.281795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399529f8-fe62-43c3-a626-b740b6a296e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:34.283220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399529f8-fe62-43c3-a626-b740b6a296e2 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:34.292535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 777/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:36.439583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50668b80-eb26-4936-b491-153c6b61f449 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:36.440203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50668b80-eb26-4936-b491-153c6b61f449 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:36.440203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50668b80-eb26-4936-b491-153c6b61f449 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:36.441579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:33:36.446984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7d51c-3a3b-4dce-86b9-8b8c619a4f0b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:36.447400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7d51c-3a3b-4dce-86b9-8b8c619a4f0b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:36.447532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7d51c-3a3b-4dce-86b9-8b8c619a4f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:36.471185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-70f7d51c-3a3b-4dce-86b9-8b8c619a4f0b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:36.472405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-70f7d51c-3a3b-4dce-86b9-8b8c619a4f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:36.480489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:36.787138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50668b80-eb26-4936-b491-153c6b61f449 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:36.788813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-50668b80-eb26-4936-b491-153c6b61f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:36.802706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50668b80-eb26-4936-b491-153c6b61f449 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:36.806215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38bd63-17ea-427b-99a2-225f68421852 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:36.806807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bd63-17ea-427b-99a2-225f68421852 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:36.807083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bd63-17ea-427b-99a2-225f68421852 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:36.807353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38bd63-17ea-427b-99a2-225f68421852 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:36.816773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bd63-17ea-427b-99a2-225f68421852 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:36.818229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38bd63-17ea-427b-99a2-225f68421852 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:36.829478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 761/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:46.464319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=63 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:46.823941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=37 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:00.017337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a380ff-5e50-4258-bf6c-777c9f079a1c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.018018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a380ff-5e50-4258-bf6c-777c9f079a1c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.018018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a380ff-5e50-4258-bf6c-777c9f079a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.053972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-66a380ff-5e50-4258-bf6c-777c9f079a1c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.055179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446e80e-8322-41ac-909e-8ba145a7fbc8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.055685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-66a380ff-5e50-4258-bf6c-777c9f079a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.056200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446e80e-8322-41ac-909e-8ba145a7fbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.056729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446e80e-8322-41ac-909e-8ba145a7fbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.071531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 779/1540] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:00.072808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0b00-b059-4481-a5b6-673bd52c0dbe None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.073119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0b00-b059-4481-a5b6-673bd52c0dbe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.073400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd0b00-b059-4481-a5b6-673bd52c0dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.092389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7446e80e-8322-41ac-909e-8ba145a7fbc8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.098638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7446e80e-8322-41ac-909e-8ba145a7fbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.105871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3ccd0b00-b059-4481-a5b6-673bd52c0dbe None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.107068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3ccd0b00-b059-4481-a5b6-673bd52c0dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.118763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 762/1541] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:00.120711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6e4cc-953b-454c-a57b-5dd81affe20e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.121296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 780/1542] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.121775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6e4cc-953b-454c-a57b-5dd81affe20e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.122280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6e4cc-953b-454c-a57b-5dd81affe20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.122495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80556a-d4de-4b68-8994-761d3b0b98dc None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.122780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80556a-d4de-4b68-8994-761d3b0b98dc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.123056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80556a-d4de-4b68-8994-761d3b0b98dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.157265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8e80556a-d4de-4b68-8994-761d3b0b98dc None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.159487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dbb6e4cc-953b-454c-a57b-5dd81affe20e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.160664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dbb6e4cc-953b-454c-a57b-5dd81affe20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.161596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8e80556a-d4de-4b68-8994-761d3b0b98dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.175619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 763/1543] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.177070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d892a-a118-4df2-9169-81848bdb32a8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.177070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d892a-a118-4df2-9169-81848bdb32a8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.177266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d892a-a118-4df2-9169-81848bdb32a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.181195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 781/1544] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.182703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.183001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.183280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.204386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.205029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1a6d892a-a118-4df2-9169-81848bdb32a8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.205652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.206347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1a6d892a-a118-4df2-9169-81848bdb32a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.214962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 764/1545] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.216290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74083e6c-4e9c-4b4b-a7f2-4b5bb6a383ba None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.216570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74083e6c-4e9c-4b4b-a7f2-4b5bb6a383ba None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.216840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74083e6c-4e9c-4b4b-a7f2-4b5bb6a383ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.219127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 782/1546] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.220052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.231868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1fd0c-4a53-4383-9476-f4273a933fde None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.237118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.237737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.238016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.238289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.238941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:00.239245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.241084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-74083e6c-4e9c-4b4b-a7f2-4b5bb6a383ba None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.241222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32f6c5-bb72-47af-a26c-6e1299c69fa4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.242428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-74083e6c-4e9c-4b4b-a7f2-4b5bb6a383ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.260331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 765/1547] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.261169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 783/1548] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:00.261751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.262030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.262301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.262524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.262935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.263217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.285833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.286973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.288076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.289289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.294803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 784/1549] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.300576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.301475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 766/1550] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.302344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.306345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f066d-d769-49e4-87cb-07feaa752f29 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.310927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.311526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.311706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8858e4-192c-4369-beec-dbd5b3d37855 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.312439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.312721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.313422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:00.313729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.315337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9b1e1-842d-4bbd-b6d5-ad6dbf5635fe admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.317195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.317795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.318079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.318354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.318997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:00.319288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.320286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:35:00.320286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:35:00.320286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:35:00.321332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f706-bd55-4377-a448-c3492af8d4fc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.326982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 785/1551] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:00.328420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.328713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.329023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.339803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 767/1552] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:00.341349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8981-3ae6-468e-afc6-f4a3bdf71b5d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.341634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8981-3ae6-468e-afc6-f4a3bdf71b5d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.341907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8981-3ae6-468e-afc6-f4a3bdf71b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.351006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.352227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.361052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 786/1553] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.361809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.362227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bfdb8981-3ae6-468e-afc6-f4a3bdf71b5d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.363414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bfdb8981-3ae6-468e-afc6-f4a3bdf71b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.371195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cd17c-8982-457c-ba20-3690ca0d16ae None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.373899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 768/1554] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.375405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.375697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.375977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.379333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.379962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.380243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.380516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.381174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.382226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:35:00.382226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:35:00.382226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:35:00.383426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.395828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.397111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.404875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 769/1555] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.405702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.423135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607647d-2e54-46a7-96cb-d0832626695d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.427681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.428278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.428554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.428825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.429523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:00.429824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.431464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0a4dd-152a-4f35-a4bb-0050f1e6f265 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.443927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 770/1556] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:00.445188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.445472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.445780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.468123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.469958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.481129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 771/1557] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.485089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.492211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10512af-8d73-4dd2-8e5d-e005b9150e71 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.496610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279f451-324a-445d-9758-9bfd0adca650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.497230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279f451-324a-445d-9758-9bfd0adca650 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.497508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279f451-324a-445d-9758-9bfd0adca650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.497777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279f451-324a-445d-9758-9bfd0adca650 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.498394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279f451-324a-445d-9758-9bfd0adca650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.501288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279f451-324a-445d-9758-9bfd0adca650 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.519516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 787/1558] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:00.519516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.580314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c72581-332b-4c2c-a50f-5a430f227620 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.585116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.590219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.590219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.590219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.590219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.590219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.660342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 772/1559] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:00.661780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f556e1-dc73-4e11-88f3-9dd4d026942b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.662068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f556e1-dc73-4e11-88f3-9dd4d026942b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.662344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f556e1-dc73-4e11-88f3-9dd4d026942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.681857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-89f556e1-dc73-4e11-88f3-9dd4d026942b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.683060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-89f556e1-dc73-4e11-88f3-9dd4d026942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.690621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 788/1560] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/projects => generated 357 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:00.691459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.748955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:00.756198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 773/1561] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.757846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.758138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.758440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.763369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31a944c-efc1-4017-873f-15b207bf9413 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.768654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.769306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.769592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.769866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.770549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:00.770854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.772510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.782754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:00.783985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:00.786276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 789/1562] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:00.787088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.792162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 774/1563] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:00.793158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.798369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf416c-1dd8-415a-b309-ff7bda08b642 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.803189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.803789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.804068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.804340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.804456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d6c5b-0d5c-4ba0-9b44-43aa9c931406 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.804999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.806892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.809079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.809716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.809993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.810263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.810893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:00.811191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.813187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.824561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 775/1564] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:00.825446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.852187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620e32a-7ccc-4c44-ad67-012e2b1ca5c1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.852187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.852187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.852187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.852684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.852684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.852684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:35:00.852684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:35:00.852684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:35:00.852684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:00.913500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 790/1565] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/projects => generated 393 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:00.914366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.953229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 776/1566] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:00.956964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:00.964132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9a525-f7b2-40c2-b002-6a006dd0c4ee admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.979452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ff7bf-80af-41b8-89ad-714f3a320395 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:00.983261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:00.984691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:00.985413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:00.985694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:00.985970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:00.986606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:00.988573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.022048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.022679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.022949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.023464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.024278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.026733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.112042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 791/1567] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.112792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.122304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81f0d2-ea3e-485d-ae46-eb10e8bcdccc admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.127057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.127770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.128052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.128323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.128982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:01.129286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.130913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.136990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 777/1568] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:00 2022] POST /identity/v3/users => generated 279 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.136990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.151623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 792/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:01.151623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.151941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ba005-9a8c-478f-b1a4-d5e03230dabb admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.152574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.153216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.153495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.153769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.154402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.155452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe4997-e5fc-4160-b1af-f848bd843b28 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.157073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.161107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.161711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.161994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.162269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.162942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:01.163268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.164950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.177714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 793/1570] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:01.178910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.192253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894522e-fe04-4578-82c8-79bff4d10a66 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.198348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.198920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.199195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.199467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.200082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.203441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.266547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 778/1571] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.267367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.300806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 794/1572] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.301768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.337286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd92a3e-2c27-4621-a397-2a9fcdbb13fa admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.364624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b8bf2-08d0-4dfa-996b-e5bca10a9208 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.369334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.369944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.370227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.370504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.371153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.372772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.375293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:01.383738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 795/1573] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.390908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.395587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e049414-3dfc-41d5-865a-259731126188 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.398621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.399274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.399558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.399835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.400290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.400471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.400897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.401183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.401459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.402079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.402571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.403692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.417552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 796/1574] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.417552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.428344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c269eb4-6ebd-4ef0-96ed-18b0d44b0bb2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.433020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.433613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.433892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.434165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.434798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.436412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.447311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 797/1575] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.448073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.457955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae510f-be79-4269-abcf-750bd65509c1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.462634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.463207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.463488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.463763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.464379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.466292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.535512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 779/1576] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.536433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.546828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b20ab-0872-4ea8-85e9-b87831d1342c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.551496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.552083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.552361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.552636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.553339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.555234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.575620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 798/1577] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.583756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.642349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e67b6-eeb3-429b-b4e5-8a6faf784670 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.647228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.647839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.648121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.648396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.649062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.650983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.682795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 780/1578] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.683677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.688836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:01.745425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568195b-e63f-4d54-ae3e-73fd264cb57e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.750519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.751158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.751436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.751784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.752373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.754387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.767798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 799/1579] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.768890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.778513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd298cf0-b55b-4a84-97e5-08a4a88ae51b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.783767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.784341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.784622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.784922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.785551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.787127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.797417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 800/1580] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.798323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.807675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e43d3-384a-4695-8036-83e0f520a1c9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.812216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.812756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.813063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.813331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.813933 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.815450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.824983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 801/1581] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.825873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.836025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e3341-6d8b-4c80-9441-3a4ce0b51914 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.840596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.841172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.841492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.841762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.842356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.844771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.854880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 802/1582] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.855766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.865315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e3b22-d791-4ec5-9573-a4a4240f3ae4 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.870021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.870611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.870889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.871159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.871766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.873328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.883133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 803/1583] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:01.883880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.893811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 781/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:01.896779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:01.896938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873da5e-3cf3-4110-b629-35956ba94ca3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.902059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.902651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.902932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.903210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.903834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.904887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5dbbb0-2caa-43cd-b488-a145a760046b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:01.905829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:01.909586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:01.910176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:01.910454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:01.910725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:01.911342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:01.913310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.047599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 804/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:02.048460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.058042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe28b4f-2814-46bf-bf23-ebdb5c439668 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.063620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.064251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.064533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.064807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.065503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.067112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.077858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 805/1586] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.078603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.088998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571d4fa-1645-4fa6-831d-04f71456be6d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.098728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 782/1587] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:01 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:02.098728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.098988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.098988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.098988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.098988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] PATH_INFO: `/v3/projects/29ede48c03e04fd6a00a874c318f5c3c/users/961ed374bc664b299c1a6425dc803dfa/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.102385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878647c1-df98-4a01-a059-393c7996ce1d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.107170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.107861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.108140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.108450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] PATH_INFO: `/v3/projects/7857413ab9f24988bde94f9bf52b76a5/users/3008efcf02c644e3bc09850d6bd6eb63/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.114505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ede48c03e04fd6a00a874c318f5c3c, user_id=961ed374bc664b299c1a6425dc803dfa, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.115720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:35:02.115720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:35:02.115720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:35:02.117660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.123788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7857413ab9f24988bde94f9bf52b76a5, user_id=3008efcf02c644e3bc09850d6bd6eb63, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.126512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.243109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 783/1588] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/7857413ab9f24988bde94f9bf52b76a5/users/3008efcf02c644e3bc09850d6bd6eb63/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:02.244040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 806/1589] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/29ede48c03e04fd6a00a874c318f5c3c/users/961ed374bc664b299c1a6425dc803dfa/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:02.244315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.247695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.257674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:02.308183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803c7d35-91d0-46ee-8282-337b6a48d99e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.312673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3caa9-8d0b-4a3e-abbd-6901091cf15c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.313833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.314497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.316436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.316436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] PATH_INFO: `/v3/projects/b49452baafa34ffc925f15234fd56e23/users/8490e6187a3344deb1f84fdb3c47b117/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.317681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.318287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.318568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.318838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.319476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.321296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.330647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49452baafa34ffc925f15234fd56e23, user_id=8490e6187a3344deb1f84fdb3c47b117, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.332986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 807/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.333686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.333824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.344493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e64592-c819-481e-93a8-eeb9daf18ca0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.349221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.350837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.350837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.350837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.351133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.352546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.363388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 808/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.364238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.375942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d07ff-5fb1-40a0-a7cd-fbe91fd0a9c1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.381360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.381958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.382238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.382559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.383171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.384716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.395251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 809/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.396035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.398703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 784/1593] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/b49452baafa34ffc925f15234fd56e23/users/8490e6187a3344deb1f84fdb3c47b117/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:02.399619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.409265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736f17-1ce6-4854-bf9d-c61491a2a42f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.441841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:02.465571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb2ab5-68bc-4e24-b810-b35b37d8beb4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.470656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70a7f2-c7c6-4db4-a99c-77a27416abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.476027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a7f2-c7c6-4db4-a99c-77a27416abcb admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.476152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423d5c5-ab7a-42df-b26e-091d77311ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.476275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a7f2-c7c6-4db4-a99c-77a27416abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.476275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70a7f2-c7c6-4db4-a99c-77a27416abcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.476461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423d5c5-ab7a-42df-b26e-091d77311ece admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.476461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423d5c5-ab7a-42df-b26e-091d77311ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.476668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70a7f2-c7c6-4db4-a99c-77a27416abcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.476762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423d5c5-ab7a-42df-b26e-091d77311ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.476762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423d5c5-ab7a-42df-b26e-091d77311ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.476977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70a7f2-c7c6-4db4-a99c-77a27416abcb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.477071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423d5c5-ab7a-42df-b26e-091d77311ece admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.487583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 810/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.487583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.487583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.487583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.488031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 785/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.489270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.489657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.490168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.506426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:02.510369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:02.510531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:02.510863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:02.515671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 811/1596] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:02.516263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.519569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 786/1597] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:02.520120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.527370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968faf8-bc43-415f-b491-888f1d275f4a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.529916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccbc6c-d67f-4024-bc78-957323bdb765 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.532996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.533659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.534071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.534329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] PATH_INFO: `/v3/projects/e94e5705bcfd476e87b2fa0af6ff597a/users/c4f4844eb5e14d789476cfb44e2516f0/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.535793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.536573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.536691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.536969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.537615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.539230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.549396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94e5705bcfd476e87b2fa0af6ff597a, user_id=c4f4844eb5e14d789476cfb44e2516f0, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.550295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 787/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.551183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.552061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.563143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51891444-5f04-45c4-8650-1d7aa65e6afd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.567737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3a101-6822-4438-9387-974ef358b473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.568321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a101-6822-4438-9387-974ef358b473 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.568601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a101-6822-4438-9387-974ef358b473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.568896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3a101-6822-4438-9387-974ef358b473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.569523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3a101-6822-4438-9387-974ef358b473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.571093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3a101-6822-4438-9387-974ef358b473 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.581254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 788/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.582530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.583069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.583359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.610844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] MFA Rules not processed for user `8490e6187a3344deb1f84fdb3c47b117`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:02.612098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:02.670006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 812/1600] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/e94e5705bcfd476e87b2fa0af6ff597a/users/c4f4844eb5e14d789476cfb44e2516f0/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:02.670800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.680613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 789/1601] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:02.681814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.692365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb1975-1591-4a85-a3a9-e475eb345ec0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.720678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:02.734542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b5c1f-69cd-47c2-80c5-bfc2553354f5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.739291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.739956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.740240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.740517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] PATH_INFO: `/v3/projects/432c7036f0c94938be4572e3bbfc118d/users/2f9a9372a0f2477b9e0833cd50609d7b/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.750459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.751076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.751357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.751638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.752278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.753922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.756980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=432c7036f0c94938be4572e3bbfc118d, user_id=2f9a9372a0f2477b9e0833cd50609d7b, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.759728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.763903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 790/1602] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:02.765462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.777464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:02.799023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:02.826697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198656fc-014a-44a8-a5cd-e922bbc0af62 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.831749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.832370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.832926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.833206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.844195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.846046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.855198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 791/1603] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:02.856148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.864541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d8e80-9ec3-4b13-a31e-2ebbcb0d58ab service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.868607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.868936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.869226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.869610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.878948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 813/1604] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/432c7036f0c94938be4572e3bbfc118d/users/2f9a9372a0f2477b9e0833cd50609d7b/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:02.879796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.881773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.884438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.897058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 792/1605] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:02.897723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:02.907296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ae1a4-004e-4a46-be96-5bc30e89057d service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.955805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92689e0d-a5f9-42d9-95a4-de5d7bcfbba4 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:02.955805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a99bd7-f278-40f8-a777-5929700edb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.955805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99bd7-f278-40f8-a777-5929700edb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.956397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99bd7-f278-40f8-a777-5929700edb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.956397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a99bd7-f278-40f8-a777-5929700edb90 admin admin] PATH_INFO: `/v3/projects/2e7330488d40469199873490c5fd2bb4/users/7b69eb143b5d42c3b4a8cb292c6a70d7/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.972975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99bd7-f278-40f8-a777-5929700edb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7330488d40469199873490c5fd2bb4, user_id=7b69eb143b5d42c3b4a8cb292c6a70d7, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.973796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:02.974489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:02.974753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:02.975031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] PATH_INFO: `/v3/projects/5544cf09578142d18e2812964499a59a/users/58c512b71af84c32b365e23b7d05ba3b/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:02.975884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a99bd7-f278-40f8-a777-5929700edb90 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:02.990826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5544cf09578142d18e2812964499a59a, user_id=58c512b71af84c32b365e23b7d05ba3b, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:02.993514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.094591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 814/1606] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/2e7330488d40469199873490c5fd2bb4/users/7b69eb143b5d42c3b4a8cb292c6a70d7/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:03.096295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.100073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.100073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.118410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.119653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.180888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 793/1607] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:02 2022] PUT /identity/v3/projects/5544cf09578142d18e2812964499a59a/users/58c512b71af84c32b365e23b7d05ba3b/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:03.180888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.201949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 815/1608] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.203653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.220705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f5b0b-10e4-4807-90cb-67b8fe1925ac None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.239400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61005271-b09e-4eb5-9672-faad50da66c9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.246731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94ba1e-0d05-4ffb-941f-3b17684206a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.248353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94ba1e-0d05-4ffb-941f-3b17684206a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.248643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94ba1e-0d05-4ffb-941f-3b17684206a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.248952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94ba1e-0d05-4ffb-941f-3b17684206a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.250092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94ba1e-0d05-4ffb-941f-3b17684206a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.252021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94ba1e-0d05-4ffb-941f-3b17684206a3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.273050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:03.289895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57e25f-c5bc-4524-83bc-9f0ba23ceb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.289895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e25f-c5bc-4524-83bc-9f0ba23ceb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.289895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e25f-c5bc-4524-83bc-9f0ba23ceb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.289895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57e25f-c5bc-4524-83bc-9f0ba23ceb59 admin admin] PATH_INFO: `/v3/projects/17e605186f7944a79d36288d99d1eaa9/users/3c7fcc99edab427aa2b750bf6b15b39c/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.303667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57e25f-c5bc-4524-83bc-9f0ba23ceb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e605186f7944a79d36288d99d1eaa9, user_id=3c7fcc99edab427aa2b750bf6b15b39c, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.306333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57e25f-c5bc-4524-83bc-9f0ba23ceb59 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.352362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 794/1609] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/projects => generated 357 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:03.353892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe0aa-24e9-4cfd-9159-2bf6472b9b10 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.354347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe0aa-24e9-4cfd-9159-2bf6472b9b10 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.355919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe0aa-24e9-4cfd-9159-2bf6472b9b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.384849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-021fe0aa-24e9-4cfd-9159-2bf6472b9b10 None None] MFA Rules not processed for user `2f9a9372a0f2477b9e0833cd50609d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.386087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-021fe0aa-24e9-4cfd-9159-2bf6472b9b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.401641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:03.455314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 795/1610] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.456657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.457002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.457285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.468475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 816/1611] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:03 2022] PUT /identity/v3/projects/17e605186f7944a79d36288d99d1eaa9/users/3c7fcc99edab427aa2b750bf6b15b39c/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:03.471204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.471605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.472004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.478787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.479943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.519918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] MFA Rules not processed for user `58c512b71af84c32b365e23b7d05ba3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.521653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.560561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 796/1612] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.561630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.624976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 817/1613] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.624976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.653934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95202e0-97d7-45c3-9e2c-79303c587b83 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.660115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085ff81-ef26-4f9b-b482-f5f5bb3e71f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.660695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085ff81-ef26-4f9b-b482-f5f5bb3e71f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.661085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085ff81-ef26-4f9b-b482-f5f5bb3e71f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.661370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085ff81-ef26-4f9b-b482-f5f5bb3e71f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.662134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085ff81-ef26-4f9b-b482-f5f5bb3e71f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.665983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085ff81-ef26-4f9b-b482-f5f5bb3e71f1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.666352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff303eb-bbbf-4e74-8ddd-d7f2a04aff11 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.679199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c56d8-4d5d-4146-81fc-ccd7ebbec69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.679199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c56d8-4d5d-4146-81fc-ccd7ebbec69a service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.679199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c56d8-4d5d-4146-81fc-ccd7ebbec69a service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.679199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c56d8-4d5d-4146-81fc-ccd7ebbec69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.695161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c56d8-4d5d-4146-81fc-ccd7ebbec69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.696675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c56d8-4d5d-4146-81fc-ccd7ebbec69a service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.707568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 797/1614] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:03.709315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ecd6-33c1-415b-9911-6c1d7a897af2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.709604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ecd6-33c1-415b-9911-6c1d7a897af2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.709879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81ecd6-33c1-415b-9911-6c1d7a897af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.732552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8c81ecd6-33c1-415b-9911-6c1d7a897af2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.734569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8c81ecd6-33c1-415b-9911-6c1d7a897af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.742454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 798/1615] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.747989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3de3e-9196-4b3e-90bf-5d2eaa9c22b6 None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.748430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3de3e-9196-4b3e-90bf-5d2eaa9c22b6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.748781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3de3e-9196-4b3e-90bf-5d2eaa9c22b6 None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.750484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 799/1616] 10.222.0.42 () {54 vars in 961 bytes} [Tue Oct 4 10:35:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:03.751236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3de3e-9196-4b3e-90bf-5d2eaa9c22b6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.763806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3de3e-9196-4b3e-90bf-5d2eaa9c22b6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.774975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 818/1617] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:03.776255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11536-abe0-4605-956e-aefab1fc9699 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.779076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11536-abe0-4605-956e-aefab1fc9699 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.779076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11536-abe0-4605-956e-aefab1fc9699 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.779413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d245c-4345-4e4c-a709-d97adc091e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.779413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d245c-4345-4e4c-a709-d97adc091e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.779595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d245c-4345-4e4c-a709-d97adc091e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.779801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d245c-4345-4e4c-a709-d97adc091e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.781166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 819/1618] 10.222.0.42 () {54 vars in 961 bytes} [Tue Oct 4 10:35:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:03.781939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11536-abe0-4605-956e-aefab1fc9699 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.792163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d245c-4345-4e4c-a709-d97adc091e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.793615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d245c-4345-4e4c-a709-d97adc091e0c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.796550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11536-abe0-4605-956e-aefab1fc9699 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.800965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.802301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.802667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.803107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.807596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 800/1619] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:03.808364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.808650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.808950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.822390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.824015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.835637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 820/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:03.836763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.851479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70612f1-f0da-4323-b402-0da2af50fb37 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.854434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.855725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.857425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58a655-d64b-4f98-b160-6e7a40b9ca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.858431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58a655-d64b-4f98-b160-6e7a40b9ca85 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.859022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58a655-d64b-4f98-b160-6e7a40b9ca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.859316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58a655-d64b-4f98-b160-6e7a40b9ca85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.860278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58a655-d64b-4f98-b160-6e7a40b9ca85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:03.862379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58a655-d64b-4f98-b160-6e7a40b9ca85 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:03.875542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 821/1621] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:03.877052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.879692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.879692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.900359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:03.901739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:03.942456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 801/1622] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.943435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:03.958220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6bbd7-6d51-44dc-8eb4-217bb4ed927f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:03.965379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:03.967080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:03.967379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:03.968264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] PATH_INFO: `/v3/projects/696674217f3243c79f51f3a1639a3e95/users/1ae8ad2ad4ef42bdbed07180ae3c0428/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:03.984271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 822/1623] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Oct 4 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:03.985547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:04.000981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=696674217f3243c79f51f3a1639a3e95, user_id=1ae8ad2ad4ef42bdbed07180ae3c0428, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.002036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f22e6-bea5-45fb-ab95-9103f39f613b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:04.008567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.011259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:04.012227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.012392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.012708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.027932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:04.056523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:04.099455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.100845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.103809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 802/1624] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:03 2022] PUT /identity/v3/projects/696674217f3243c79f51f3a1639a3e95/users/1ae8ad2ad4ef42bdbed07180ae3c0428/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:04.105888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:04.119896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 823/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:04.133826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:04.148019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392d93a-8bd3-45cc-a996-b4f25c841456 service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:04.221913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:04.222816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.223286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.223709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.224594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.226548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.248095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 824/1626] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:04.252964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:04.267065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e25834-963e-4d91-9833-240f2f2776a3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:04.273468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e702409-5c7c-4152-9d92-5659bbabc3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:04.274156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e702409-5c7c-4152-9d92-5659bbabc3ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.274444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e702409-5c7c-4152-9d92-5659bbabc3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.274723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e702409-5c7c-4152-9d92-5659bbabc3ad admin admin] PATH_INFO: `/v3/projects/696674217f3243c79f51f3a1639a3e95/users/1ae8ad2ad4ef42bdbed07180ae3c0428/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.275054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4d819-b419-4281-98ff-e7a181bbcf4d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:04.277497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e702409-5c7c-4152-9d92-5659bbabc3ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=696674217f3243c79f51f3a1639a3e95, user_id=1ae8ad2ad4ef42bdbed07180ae3c0428, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.281626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e702409-5c7c-4152-9d92-5659bbabc3ad admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.284221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ba05d8-474f-4404-8d2c-7c520ffaa827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:04.284999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba05d8-474f-4404-8d2c-7c520ffaa827 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.285379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba05d8-474f-4404-8d2c-7c520ffaa827 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.285747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba05d8-474f-4404-8d2c-7c520ffaa827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.299130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 825/1627] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:04 2022] PUT /identity/v3/projects/696674217f3243c79f51f3a1639a3e95/users/1ae8ad2ad4ef42bdbed07180ae3c0428/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:04.309135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.309405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.309682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.348225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] MFA Rules not processed for user `1ae8ad2ad4ef42bdbed07180ae3c0428`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:04.352568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:04.365508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba05d8-474f-4404-8d2c-7c520ffaa827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.367121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ba05d8-474f-4404-8d2c-7c520ffaa827 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.379096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 803/1628] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:04.481775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a0cf-a439-4cc8-9f7c-178775a5d36c None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.482059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a0cf-a439-4cc8-9f7c-178775a5d36c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.482339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a0cf-a439-4cc8-9f7c-178775a5d36c None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.486624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 804/1629] 10.222.0.42 () {54 vars in 912 bytes} [Tue Oct 4 10:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:35:04.493452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.493720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.493994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.514507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 826/1630] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:04.518318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:04.519021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:04.526886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:04.647055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c1178-d672-41cf-9b34-ded400a7cd0c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:04.651299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425d65-7299-40d6-a5d8-9e1223d5c7ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:04.651924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425d65-7299-40d6-a5d8-9e1223d5c7ee service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.652209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425d65-7299-40d6-a5d8-9e1223d5c7ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.652507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425d65-7299-40d6-a5d8-9e1223d5c7ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.661372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 805/1631] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:35:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:04.673018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425d65-7299-40d6-a5d8-9e1223d5c7ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.673018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425d65-7299-40d6-a5d8-9e1223d5c7ee service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.674390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:04.684132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 827/1632] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:35:04.692677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e862e-29f0-4a29-a4f3-7ec70bd3be65 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:04.698127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c32e15-6a66-4ec4-8286-39e0aad5165e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:04.698763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c32e15-6a66-4ec4-8286-39e0aad5165e service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.699733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c32e15-6a66-4ec4-8286-39e0aad5165e service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.700502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c32e15-6a66-4ec4-8286-39e0aad5165e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.730108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c32e15-6a66-4ec4-8286-39e0aad5165e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:04.731931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c32e15-6a66-4ec4-8286-39e0aad5165e service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:04.745524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 806/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:35:04.793687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7adc1-9206-449f-a3da-539b79babd16 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.793968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7adc1-9206-449f-a3da-539b79babd16 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.794242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7adc1-9206-449f-a3da-539b79babd16 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.795744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 828/1634] 10.222.0.42 () {54 vars in 912 bytes} [Tue Oct 4 10:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:04.809383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:04.809383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:04.809383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:04.837632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:04.843616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:04.868602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 807/1635] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:35:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:04.883548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:05.023188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7adc1-9206-449f-a3da-539b79babd16 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:05.037955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7adc1-9206-449f-a3da-539b79babd16 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:05.042702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:05.043494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:05.043922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:05.044196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:05.061353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:05.063419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:05.098247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 829/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:05.474878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:06.168439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:06.183641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df4796-375d-4020-b6aa-9196485b8948 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:06.194836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afe26f-a42e-49fd-857e-aae4e7833e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:06.195477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afe26f-a42e-49fd-857e-aae4e7833e69 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:06.195805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afe26f-a42e-49fd-857e-aae4e7833e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:06.196117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afe26f-a42e-49fd-857e-aae4e7833e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:06.338596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afe26f-a42e-49fd-857e-aae4e7833e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:06.341343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afe26f-a42e-49fd-857e-aae4e7833e69 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:06.363895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 808/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:10.867760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:11.079178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101da37-a4c2-453f-acf9-a3fb06f1da66 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:11.090028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:11.091722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:11.094785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:11.095260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] PATH_INFO: `/v3/users/58c512b71af84c32b365e23b7d05ba3b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:11.097473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c512b71af84c32b365e23b7d05ba3b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:11.099897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: WARNING py.warnings [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:35:11.099897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: warnings.warn(msg) Oct 04 10:35:11.099897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: Oct 04 10:35:11.103315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:11.486215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c512b71af84c32b365e23b7d05ba3b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:11.490705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c512b71af84c32b365e23b7d05ba3b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:11.589970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c512b71af84c32b365e23b7d05ba3b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:11.595533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c512b71af84c32b365e23b7d05ba3b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:11.734055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c512b71af84c32b365e23b7d05ba3b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:11.790459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 830/1638] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:10 2022] DELETE /identity/v3/users/58c512b71af84c32b365e23b7d05ba3b => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:11.795920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:11.796464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:11.796680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:11.844701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:11.846033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:12.046206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 809/1639] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:12.062274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:12.064165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:12.150096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:12.206884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e3bf4-e739-4098-9019-107c6047b8ef admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:12.218057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267e6de-d9fb-45ff-ac26-31571bfd5fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:12.218811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267e6de-d9fb-45ff-ac26-31571bfd5fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:12.219142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267e6de-d9fb-45ff-ac26-31571bfd5fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:12.219495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267e6de-d9fb-45ff-ac26-31571bfd5fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:12.249665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267e6de-d9fb-45ff-ac26-31571bfd5fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:12.251564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267e6de-d9fb-45ff-ac26-31571bfd5fb1 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:12.263708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 831/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:14.819049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:14.819049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:14.837795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b52c9c-9738-4aad-be11-9ab97bfe9dd5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:14.847646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:14.849282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:14.850006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:14.850501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] PATH_INFO: `/v3/projects/5544cf09578142d18e2812964499a59a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:14.853330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5544cf09578142d18e2812964499a59a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:14.855866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: WARNING py.warnings [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:35:14.855866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: warnings.warn(msg) Oct 04 10:35:14.855866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: Oct 04 10:35:14.859493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:15.035831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:15.079451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5544cf09578142d18e2812964499a59a', 'request_id': 'req-0e99cabb-2783-46ed-a981-86252c0ab4a1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:15.115154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5544cf09578142d18e2812964499a59a', 'request_id': 'req-0e99cabb-2783-46ed-a981-86252c0ab4a1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:15.142567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5544cf09578142d18e2812964499a59a', 'request_id': 'req-0e99cabb-2783-46ed-a981-86252c0ab4a1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:15.253994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] The token cache is being invalidate because project 5544cf09578142d18e2812964499a59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:15.254516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:15.257552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0e99cabb-2783-46ed-a981-86252c0ab4a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:15.267545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 810/1641] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:14 2022] DELETE /identity/v3/projects/5544cf09578142d18e2812964499a59a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:15.268676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:15.269180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:15.269573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:15.313069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:15.317948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:15.470466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 832/1642] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:15.478029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:15.478324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:15.478607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:15.550487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:15.560581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:15.589640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 811/1643] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:15.595021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:15.615290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea37f6-06d3-4f72-a4a0-f44d50a52a2e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:15.620714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:15.621489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:15.621867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:15.622221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:15.623116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:15.623579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:15.626752 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:15.645136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 833/1644] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:35:15.652032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:15.676301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e56d7a-559f-41d2-8c51-f8dd03cb3cc1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:15.698710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:15.699773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:15.700131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:15.701574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:15.705871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:15.710192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:15.863606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:15.874338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 812/1645] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:15 2022] POST /identity/v3/projects => generated 363 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:15.880639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f0c5d-58ee-4572-8e20-86a6994c76d5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:16.009690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:16.010700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.010700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.010881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.014507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:16.022917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:16.159340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 834/1646] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:15 2022] POST /identity/v3/users => generated 282 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:16.164048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:16.177166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423f94-87e0-471e-843a-e50493e325ef admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:16.185041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:16.186932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.187444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.187921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.190275 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:16.192172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:16.228000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:16.228614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 813/1647] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:16.240653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f825c-ebfe-4fe5-a046-704595c56d5b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:16.257120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe8b61-4b7a-47d3-b818-1f4ea6dc3e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:16.257939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe8b61-4b7a-47d3-b818-1f4ea6dc3e79 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.258398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe8b61-4b7a-47d3-b818-1f4ea6dc3e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.264054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe8b61-4b7a-47d3-b818-1f4ea6dc3e79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.264054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe8b61-4b7a-47d3-b818-1f4ea6dc3e79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:16.265552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe8b61-4b7a-47d3-b818-1f4ea6dc3e79 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:16.292812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 835/1648] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:16.295324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:16.311706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e200b0c-9262-4a16-b51f-e15e1c3c85ef admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:16.330987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd25d3-5d93-4551-a111-5823eece31d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:16.331820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd25d3-5d93-4551-a111-5823eece31d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.332110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd25d3-5d93-4551-a111-5823eece31d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.332392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd25d3-5d93-4551-a111-5823eece31d7 admin admin] PATH_INFO: `/v3/projects/6d8793c7d0fe4e1997050e8c5efd599c/users/caf76275d01f44279f086f427dca6f6f/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.374896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd25d3-5d93-4551-a111-5823eece31d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8793c7d0fe4e1997050e8c5efd599c, user_id=caf76275d01f44279f086f427dca6f6f, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:16.379451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd25d3-5d93-4551-a111-5823eece31d7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:16.486667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.486789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.487023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.523694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 814/1649] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:16 2022] PUT /identity/v3/projects/6d8793c7d0fe4e1997050e8c5efd599c/users/caf76275d01f44279f086f427dca6f6f/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:16.535351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.535639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.535922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.545696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] MFA Rules not processed for user `961ed374bc664b299c1a6425dc803dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:16.558653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:16.568804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] MFA Rules not processed for user `caf76275d01f44279f086f427dca6f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:16.570163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:16.615962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:16.712086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 836/1650] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:16.720437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:16.722776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:16.781495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 815/1651] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:16.786935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:16.883216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f2b08-87f7-4605-8c42-696488bdb91b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:16.892255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:16.893251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.893524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.893903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.907649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:16.911150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:16.915608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:16.933373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 837/1652] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:35:16.945259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6ac7a-e0ef-478f-9112-c32816aaf747 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:16.960071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:16.960727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:16.961046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:16.961357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:16.962082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:16.976266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:17.158429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 816/1653] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:16 2022] POST /identity/v3/projects => generated 365 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:17.159415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:17.190330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a71d6-86dd-489e-9185-7dc136af0921 service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:17.350387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:17.352078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:17.352593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:17.353155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:17.355476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:17.366039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:17.543856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 838/1654] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:17 2022] POST /identity/v3/users => generated 289 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:17.546900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:17.580261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059d08-ba25-4ad3-a053-bf4be2a11197 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:17.590831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:17.592185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:17.592735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:17.597758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:17.599024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:17.601919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:17.647066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 817/1655] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:17.650971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:17.671957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642262-ae1c-467e-b355-b5fe7a46cdc5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:17.704995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:17.706247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:17.706247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:17.706247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] PATH_INFO: `/v3/projects/daec1a71764640c8a690f1e7cb4f31d4/users/ec942dc4a42d41ee8d21df2e705307c1/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:17.752431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daec1a71764640c8a690f1e7cb4f31d4, user_id=ec942dc4a42d41ee8d21df2e705307c1, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:17.758600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:17.908667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 839/1656] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:17 2022] PUT /identity/v3/projects/daec1a71764640c8a690f1e7cb4f31d4/users/ec942dc4a42d41ee8d21df2e705307c1/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:17.912508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:17.929682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911a26f-17ef-4a49-9b2d-144b35c1faa5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:18.051764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:18.051764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:18.121808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:18.123092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:18.124359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:18.124778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:18.126179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:18.132299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:18.169468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 818/1657] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:18.172343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:18.198403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bef74-21b4-44f3-8ca6-c57a3c65c2fb admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:18.211465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e044138-a6ca-4144-af8d-027978762a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:18.212938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e044138-a6ca-4144-af8d-027978762a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:18.212938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e044138-a6ca-4144-af8d-027978762a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:18.212938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e044138-a6ca-4144-af8d-027978762a98 admin admin] PATH_INFO: `/v3/projects/daec1a71764640c8a690f1e7cb4f31d4/users/ec942dc4a42d41ee8d21df2e705307c1/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:18.217982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e044138-a6ca-4144-af8d-027978762a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daec1a71764640c8a690f1e7cb4f31d4, user_id=ec942dc4a42d41ee8d21df2e705307c1, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:18.220482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2d39-8138-4b7d-90eb-bb3fc2911ba5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:18.224591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e044138-a6ca-4144-af8d-027978762a98 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:18.231204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:18.240832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:18.241638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:18.241933 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:18.242392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] PATH_INFO: `/v3/users/2f9a9372a0f2477b9e0833cd50609d7b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:18.244471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9a9372a0f2477b9e0833cd50609d7b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:18.249226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:18.285199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 819/1658] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:18 2022] PUT /identity/v3/projects/daec1a71764640c8a690f1e7cb4f31d4/users/ec942dc4a42d41ee8d21df2e705307c1/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:18.286931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:18.291958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:18.291958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:18.392254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] MFA Rules not processed for user `ec942dc4a42d41ee8d21df2e705307c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:18.392254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:18.541287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9a9372a0f2477b9e0833cd50609d7b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:18.542071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9a9372a0f2477b9e0833cd50609d7b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:18.556424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9a9372a0f2477b9e0833cd50609d7b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:18.558127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9a9372a0f2477b9e0833cd50609d7b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:18.593286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 820/1659] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:18.594727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:18.600526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:18.637125 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ca2dea23-211c-44c7-9290-cf573ff1c984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9a9372a0f2477b9e0833cd50609d7b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:18.691908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 840/1660] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:18 2022] DELETE /identity/v3/users/2f9a9372a0f2477b9e0833cd50609d7b => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:18.699822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:18.702283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:18.705676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:18.759442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58762b1-b84e-453a-b3ec-dc3aa0f3d892 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:18.771511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:18.775432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:18.941670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:18.943944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:18.944403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:18.944913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:18.947853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:18.952663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 841/1661] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:19.013476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:19.015767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:19.137621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5b357-1714-4f9d-b2fa-a02e58b3a147 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:19.183009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:19.183009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:19.183009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:19.183009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:19.183314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:19.183314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8339212-9866-4b17-8380-a96c48215400 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:19.183447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:19.183447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:19.193306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 821/1662] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:19.205450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 842/1663] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:35:19.611892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:19.611892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:19.611892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:19.660772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] MFA Rules not processed for user `3008efcf02c644e3bc09850d6bd6eb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:19.663005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:19.689774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:19.783043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:19.785378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:19.815821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c4197-f5e9-4790-923d-100e3d91156b service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:19.828387 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:19.830586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:19.830866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:19.835967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:19.838800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:19.847107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:19.871682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 822/1664] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:19.894134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:19.896299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:20.001691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:20.004754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad05ffc-4f71-4a89-8a33-8607346d1cf6 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:20.029442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 843/1665] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:20.074206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d8b88f-c117-43cf-ae3e-2d6b5a9d426d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:20.086424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:20.093096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:20.093096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:20.093096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:20.114490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:20.117017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:20.133233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 823/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:20.351226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29607265-1d53-447d-9669-6332984c0c54 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:20.351519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29607265-1d53-447d-9669-6332984c0c54 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:20.351781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29607265-1d53-447d-9669-6332984c0c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:20.429362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-29607265-1d53-447d-9669-6332984c0c54 None None] MFA Rules not processed for user `7b69eb143b5d42c3b4a8cb292c6a70d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:20.431339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-29607265-1d53-447d-9669-6332984c0c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:20.461047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:20.461047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:20.491402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17a283-8789-48e7-bc2a-9a6cb1da5aaa service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:20.519989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:20.520734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:20.521144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:20.521714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] PATH_INFO: `/v3/projects/432c7036f0c94938be4572e3bbfc118d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:20.526106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=432c7036f0c94938be4572e3bbfc118d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:20.533650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:20.596300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 844/1667] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:20.610385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:20.613247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:20.615537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:20.706182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] MFA Rules not processed for user `7b69eb143b5d42c3b4a8cb292c6a70d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:20.707448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:20.730079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 845/1668] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:20.739003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:20.741721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:20.817616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432c7036f0c94938be4572e3bbfc118d', 'request_id': 'req-ed90a029-9459-4ede-8f8c-60a8e996cc4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:20.825406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432c7036f0c94938be4572e3bbfc118d', 'request_id': 'req-ed90a029-9459-4ede-8f8c-60a8e996cc4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:20.829094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab635b-b837-483f-ac8f-490b16639073 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:20.848794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432c7036f0c94938be4572e3bbfc118d', 'request_id': 'req-ed90a029-9459-4ede-8f8c-60a8e996cc4c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:20.965707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] The token cache is being invalidate because project 432c7036f0c94938be4572e3bbfc118d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:20.966048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:20.967008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ed90a029-9459-4ede-8f8c-60a8e996cc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:20.978325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 824/1669] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:20 2022] DELETE /identity/v3/projects/432c7036f0c94938be4572e3bbfc118d => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:20.981782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec218f1-7f40-4637-8532-69525a732f3e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:20.982297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec218f1-7f40-4637-8532-69525a732f3e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:20.982840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec218f1-7f40-4637-8532-69525a732f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:20.992475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:20.994025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:20.994428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:20.994774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.000248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:21.023473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4ec218f1-7f40-4637-8532-69525a732f3e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:21.026305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4ec218f1-7f40-4637-8532-69525a732f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:21.122949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 825/1670] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:21.133701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:21.137917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:21.139542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.160272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:21.162772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:21.171759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:21.173282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:21.174593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 846/1671] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:21.182708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 826/1672] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:21.189354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:21.208626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619339e7-4d58-4175-80c6-2aadedaad39b service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:21.220250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:21.221588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:21.231346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:21.235500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.235874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:21.236600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:21.243054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:21.262758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 847/1673] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:21.263343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:21.284077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29bdca-492e-4a97-9c1a-10075d85eec8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:21.293535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:21.294184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:21.294468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:21.294820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.295479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:21.297519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:21.438726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:21.479775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 827/1674] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:21 2022] POST /identity/v3/projects => generated 363 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:21.486063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:21.509500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fae6a-1f96-4d02-aa46-475ab5de6aef admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:21.665151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:21.672841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:21.680586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:21.685883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.685883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:21.685883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:21.795159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 848/1675] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:21 2022] POST /identity/v3/users => generated 282 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:21.799128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:21.814296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da7da4-079a-403e-b284-150e65edc55a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:21.835176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:21.848743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:21.849478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:21.850168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.852795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:21.859425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:21.895937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:21.896751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 828/1676] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:21.912970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbd0a8-e3db-40af-993c-912932af554d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:21.937124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75da0b-0eef-4e12-ba87-2efed9e93145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:21.937756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75da0b-0eef-4e12-ba87-2efed9e93145 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:21.938218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75da0b-0eef-4e12-ba87-2efed9e93145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:21.938668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75da0b-0eef-4e12-ba87-2efed9e93145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:21.939455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75da0b-0eef-4e12-ba87-2efed9e93145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:21.941251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75da0b-0eef-4e12-ba87-2efed9e93145 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:21.997892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:22.001076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 849/1677] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:22.006785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:22.042876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4179-9d17-4a1e-bdef-638498c2427c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:22.074333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acebec-07c5-4f82-ba0a-ee25d3c422de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:22.075512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acebec-07c5-4f82-ba0a-ee25d3c422de admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:22.075901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acebec-07c5-4f82-ba0a-ee25d3c422de admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:22.077829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acebec-07c5-4f82-ba0a-ee25d3c422de admin admin] PATH_INFO: `/v3/projects/553cbf99aa0e4957ae876da56ff6f45e/users/25500227a1914a51a4d50dd58e6f978d/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:22.134682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:22.181507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acebec-07c5-4f82-ba0a-ee25d3c422de admin admin] RBAC: Authorizing `identity:create_grant(project_id=553cbf99aa0e4957ae876da56ff6f45e, user_id=25500227a1914a51a4d50dd58e6f978d, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:22.200748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8451-46da-4f24-81e2-be8eea00098e None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:22.200748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8451-46da-4f24-81e2-be8eea00098e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:22.200748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf8451-46da-4f24-81e2-be8eea00098e None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:22.200748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 850/1678] 10.222.0.42 () {54 vars in 912 bytes} [Tue Oct 4 10:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:35:22.202056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acebec-07c5-4f82-ba0a-ee25d3c422de admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:22.205788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:22.206080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:22.206359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:22.358918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 829/1679] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:22 2022] PUT /identity/v3/projects/553cbf99aa0e4957ae876da56ff6f45e/users/25500227a1914a51a4d50dd58e6f978d/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:22.361065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12443d-a749-4898-bf48-e6831485887c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:22.361606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12443d-a749-4898-bf48-e6831485887c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:22.362086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12443d-a749-4898-bf48-e6831485887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:22.405513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:22.410019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:22.481730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-7d12443d-a749-4898-bf48-e6831485887c None None] MFA Rules not processed for user `25500227a1914a51a4d50dd58e6f978d`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:22.483757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7d12443d-a749-4898-bf48-e6831485887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:22.683366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 851/1680] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:22.697569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:22.703874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:22.732785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a4217-beb8-4be1-9af2-ae61b79a214e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:22.733503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 830/1681] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:22.734631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12443d-a749-4898-bf48-e6831485887c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:22.745767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:22.746662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:22.747085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:22.747499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:22.775163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:22.781431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:22.818590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 852/1682] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:35:22.832444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:22.866099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12443d-a749-4898-bf48-e6831485887c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:22.882169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:22.885900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:22.885900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:22.885900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:22.885900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:22.885900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:23.092288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 831/1683] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:22 2022] POST /identity/v3/projects => generated 363 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:23.093090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:23.120218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865b08d-87be-4506-9be1-931036e8ddca service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:23.328732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:23.329518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.329676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.329953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.330595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:23.337394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:23.466230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 853/1684] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:23 2022] POST /identity/v3/users => generated 288 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:23.471321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:23.500084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66bcb9e-a56b-403b-8f2c-9e9980509de8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:23.512989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:23.515386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.515853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.516336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.516981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:23.521838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:23.554760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:23.555799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 832/1685] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:23.570489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e4d14-c7de-4952-af73-90e44cd9b194 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:23.591396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588edc93-f345-4762-8413-9aad0f82a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:23.593599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588edc93-f345-4762-8413-9aad0f82a9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.593906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588edc93-f345-4762-8413-9aad0f82a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.594181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588edc93-f345-4762-8413-9aad0f82a9ac admin admin] PATH_INFO: `/v3/projects/6ac4979497254fc48e1bd7b47ff46b02/users/0b74b4dfa5de45579f047c02501438e4/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.637986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588edc93-f345-4762-8413-9aad0f82a9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac4979497254fc48e1bd7b47ff46b02, user_id=0b74b4dfa5de45579f047c02501438e4, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:23.642176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588edc93-f345-4762-8413-9aad0f82a9ac admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:23.722900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 854/1686] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:23 2022] PUT /identity/v3/projects/6ac4979497254fc48e1bd7b47ff46b02/users/0b74b4dfa5de45579f047c02501438e4/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:23.727051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:23.739386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a686715-ffba-4bd7-9ae9-e31b2520d34e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:23.742762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.743216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.744065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.807629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] MFA Rules not processed for user `3c7fcc99edab427aa2b750bf6b15b39c`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:23.809186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:23.822321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:23.873308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bae1f0-47eb-42c1-a592-abb3885c3c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:23.874422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bae1f0-47eb-42c1-a592-abb3885c3c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.874709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bae1f0-47eb-42c1-a592-abb3885c3c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.874973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bae1f0-47eb-42c1-a592-abb3885c3c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.875621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bae1f0-47eb-42c1-a592-abb3885c3c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:23.877343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bae1f0-47eb-42c1-a592-abb3885c3c6e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:23.903723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 833/1687] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:23.905270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e807df-006c-4e6d-a136-c65d90f7f2d9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.905791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e807df-006c-4e6d-a136-c65d90f7f2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.906205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e807df-006c-4e6d-a136-c65d90f7f2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.916487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 855/1688] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:23.917592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:23.942687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77981e-942d-427d-98df-f01645d04d7c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:23.950041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:23.951923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:23.952313 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:23.952764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] PATH_INFO: `/v3/projects/6ac4979497254fc48e1bd7b47ff46b02/users/0b74b4dfa5de45579f047c02501438e4/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:23.955153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac4979497254fc48e1bd7b47ff46b02, user_id=0b74b4dfa5de45579f047c02501438e4, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:23.958785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:23.972649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 856/1689] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:23 2022] PUT /identity/v3/projects/6ac4979497254fc48e1bd7b47ff46b02/users/0b74b4dfa5de45579f047c02501438e4/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:23.973976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:23.976327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:23.984373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-72e807df-006c-4e6d-a136-c65d90f7f2d9 None None] MFA Rules not processed for user `c4f4844eb5e14d789476cfb44e2516f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:23.986165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-72e807df-006c-4e6d-a136-c65d90f7f2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.054623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:24.133527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 834/1690] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:24.147350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97bb3c-f4a1-4813-8347-95c38de5b7f7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:24.152251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97bb3c-f4a1-4813-8347-95c38de5b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:24.152809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97bb3c-f4a1-4813-8347-95c38de5b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:24.159561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e76a11-566a-4ffb-bbfc-f483ef0a5023 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:24.192463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:24.198937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:24.200904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:24.201197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:24.209893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4b97bb3c-f4a1-4813-8347-95c38de5b7f7 None None] MFA Rules not processed for user `0b74b4dfa5de45579f047c02501438e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:24.211698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4b97bb3c-f4a1-4813-8347-95c38de5b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.347854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 835/1691] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:24.362644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:24.363611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:24.364083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:24.394563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:24.396914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:24.405499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] MFA Rules not processed for user `c4f4844eb5e14d789476cfb44e2516f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:24.407287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.416726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 836/1692] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:24.418851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:24.422165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.427105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 857/1693] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:24.428091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:24.429940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.595353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301cef9-f115-49c2-a6e2-a66ecbac46d1 service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:24.602445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7577d-7cf1-4440-9a19-332be155f238 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:24.604689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:24.606551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:24.606778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:24.607059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:24.615001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:24.616586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:24.619043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:24.619378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:24.624688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.627033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:24.629768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:24.654948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:24.655106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 858/1694] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:24.655106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:24.655256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63b7a3-fe0c-400c-b89f-4f4ff872b44c service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:24.657678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:24.677327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 837/1695] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:24.685440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be8fc-7bde-420c-b313-960ba5bb28f2 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:24.700370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:24.701621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:24.701621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:24.701900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:24.725893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:24.729328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:24.774003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 859/1696] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:25.022168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:25.022617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:25.023263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:25.081238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] MFA Rules not processed for user `961ed374bc664b299c1a6425dc803dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:25.086579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:25.246900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 838/1697] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:25.347971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:25.362678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923488a-ad7f-4211-a70a-bc370697aebd service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:25.387831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78908b5-a905-4e49-b39e-e41a0f9ffbd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:25.388647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78908b5-a905-4e49-b39e-e41a0f9ffbd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:25.389661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78908b5-a905-4e49-b39e-e41a0f9ffbd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:25.390426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78908b5-a905-4e49-b39e-e41a0f9ffbd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:25.531649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78908b5-a905-4e49-b39e-e41a0f9ffbd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:25.534314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78908b5-a905-4e49-b39e-e41a0f9ffbd0 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:25.559024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 860/1698] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:25.666481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:25.901137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:25.912162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:26.055120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f010b-7b12-4f13-9dc1-d5a6dad7beee None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:26.064125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:26.065115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:26.065546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:26.072957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] PATH_INFO: `/v3/users/caf76275d01f44279f086f427dca6f6f` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:26.072957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf76275d01f44279f086f427dca6f6f)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:26.072957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:26.125000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:26.266012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1ab7c-8f46-44de-b571-9a6eba77b3e7 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:26.266295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1ab7c-8f46-44de-b571-9a6eba77b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:26.266912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1ab7c-8f46-44de-b571-9a6eba77b3e7 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:26.270423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 861/1699] 10.222.0.42 () {54 vars in 912 bytes} [Tue Oct 4 10:35:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:26.275837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:26.276124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:26.276454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:26.323429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:26.324691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:26.433145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf76275d01f44279f086f427dca6f6f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:26.437420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf76275d01f44279f086f427dca6f6f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:26.453508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 862/1700] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:26.462288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf76275d01f44279f086f427dca6f6f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:26.465120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf76275d01f44279f086f427dca6f6f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:26.484126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:26.500804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:26.533258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408902f-1b9d-4687-ab27-8bba5bcd7e13 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:26.539174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:26.540203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:26.541290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:26.541842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:26.544591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:26.562434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:26.564782 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d034e-1be6-4495-b141-34abbe1979ec service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:26.579166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 863/1701] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:26.654505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ac3e308e-70a4-40af-9e93-67be9b43a277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf76275d01f44279f086f427dca6f6f'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:26.690797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 839/1702] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:25 2022] DELETE /identity/v3/users/caf76275d01f44279f086f427dca6f6f => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:26.692896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2d97-16f7-4133-8080-f547f234eb57 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:26.693290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2d97-16f7-4133-8080-f547f234eb57 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:26.693690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a2d97-16f7-4133-8080-f547f234eb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:26.731081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:26.731636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:26.732013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:26.764907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f34a2d97-16f7-4133-8080-f547f234eb57 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:26.774614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f34a2d97-16f7-4133-8080-f547f234eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:26.811038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] MFA Rules not processed for user `3008efcf02c644e3bc09850d6bd6eb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:26.812693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:27.022100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 840/1703] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:27.063077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 864/1704] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:27.075108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:27.077282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:27.106307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:27.316746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e2a8c-78dd-4458-a10d-3c031ffe054d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:27.333179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:27.333993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:27.334419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:27.340073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:27.368557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:27.374399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:27.393445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:27.393750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:27.394037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:27.397140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 841/1705] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:27.470022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] MFA Rules not processed for user `c4f4844eb5e14d789476cfb44e2516f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:27.473188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:27.702073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 865/1706] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:29.530380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:29.538071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:29.591923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba788ec-caf8-4af4-99e7-03b90ab5d536 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:29.650525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:29.655173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:29.655871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:29.656171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] PATH_INFO: `/v3/projects/6d8793c7d0fe4e1997050e8c5efd599c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:29.659420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8793c7d0fe4e1997050e8c5efd599c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:29.669566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:29.880816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8793c7d0fe4e1997050e8c5efd599c', 'request_id': 'req-3e5c437d-329e-4705-a810-5a20eaf34487'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:29.911381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8793c7d0fe4e1997050e8c5efd599c', 'request_id': 'req-3e5c437d-329e-4705-a810-5a20eaf34487'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:29.929634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8793c7d0fe4e1997050e8c5efd599c', 'request_id': 'req-3e5c437d-329e-4705-a810-5a20eaf34487'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.031425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] The token cache is being invalidate because project 6d8793c7d0fe4e1997050e8c5efd599c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:30.032215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.039986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3e5c437d-329e-4705-a810-5a20eaf34487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.039986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 842/1707] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:29 2022] DELETE /identity/v3/projects/6d8793c7d0fe4e1997050e8c5efd599c => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:30.043414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:30.062757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a5b3e-17ab-415e-8274-1ed336195763 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:30.158936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:30.221754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:30.222804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:30.224191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:30.224675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] PATH_INFO: `/v3/users/ec942dc4a42d41ee8d21df2e705307c1` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:30.226189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec942dc4a42d41ee8d21df2e705307c1)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:30.229066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:30.566406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec942dc4a42d41ee8d21df2e705307c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.568419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec942dc4a42d41ee8d21df2e705307c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.607231 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec942dc4a42d41ee8d21df2e705307c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.610668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec942dc4a42d41ee8d21df2e705307c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.717840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe4f0e22-68cb-406c-85a3-1a5d1b5e09d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec942dc4a42d41ee8d21df2e705307c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:30.798523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 866/1708] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:30 2022] DELETE /identity/v3/users/ec942dc4a42d41ee8d21df2e705307c1 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:31.396144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:31.396724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:31.396724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:31.451954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] MFA Rules not processed for user `3c7fcc99edab427aa2b750bf6b15b39c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:31.465226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:31.526673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:31.533303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:31.533303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:31.568643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:31.652260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 843/1709] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:31.802976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:31.807770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:32.024149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 867/1710] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:35:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:32.393308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:32.408495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:32.477744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:32.479907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:32.524633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caaef5-8d57-4972-814b-89bab92e1899 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:32.539187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:32.540062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:32.540452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:32.541851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] PATH_INFO: `/v3/projects/daec1a71764640c8a690f1e7cb4f31d4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:32.546007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daec1a71764640c8a690f1e7cb4f31d4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:32.549474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:32.549601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:32.648663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464a77d-de13-4e0c-bdfb-8aca3d726b5b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:32.656784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f569989-986c-4c3d-9231-d113577cb370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:32.657833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f569989-986c-4c3d-9231-d113577cb370 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:32.658327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f569989-986c-4c3d-9231-d113577cb370 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:32.660010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f569989-986c-4c3d-9231-d113577cb370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:32.672349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f569989-986c-4c3d-9231-d113577cb370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:32.676407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f569989-986c-4c3d-9231-d113577cb370 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:32.687121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 868/1711] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:35:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:35:32.779598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daec1a71764640c8a690f1e7cb4f31d4', 'request_id': 'req-1cd5d905-1bc9-4119-b50e-a4f86f1df631'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:32.797905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daec1a71764640c8a690f1e7cb4f31d4', 'request_id': 'req-1cd5d905-1bc9-4119-b50e-a4f86f1df631'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:32.821651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daec1a71764640c8a690f1e7cb4f31d4', 'request_id': 'req-1cd5d905-1bc9-4119-b50e-a4f86f1df631'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:32.911495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] The token cache is being invalidate because project daec1a71764640c8a690f1e7cb4f31d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:32.911955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:32.912888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1cd5d905-1bc9-4119-b50e-a4f86f1df631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:32.915403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 844/1712] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:32 2022] DELETE /identity/v3/projects/daec1a71764640c8a690f1e7cb4f31d4 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:32.920264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:32.922864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:32.922864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:32.957296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:32.958494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:33.134175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 869/1713] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:33.136649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.136969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.137247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.164800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:33.166566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:33.193774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 845/1714] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:33.197852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:33.212092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdb403-cacd-4253-9346-f2284de820fd None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:33.225266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:33.226230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.226959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.227559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.228398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:33.228913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:33.230725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:33.261789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 870/1715] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:35:33.266418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:33.276419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1438e-7469-43bf-8855-5c837ea3af61 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:33.283287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:33.285260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.289164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.289564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.295753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:33.295753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:33.446841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 846/1716] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:33 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:33.451947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:33.475547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e8e31-1484-4b10-9b88-b93e9381d7a5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:33.561320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:33.561974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.562205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.562683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.563517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:33.565692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:33.775523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 871/1717] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:33 2022] POST /identity/v3/users => generated 294 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:33.778119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:33.791943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713f9e3-ca7f-4bd9-bdff-5f561848405b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:33.799685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:33.800383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.800749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.801122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.801998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:33.804887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:33.821359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:33.827033 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 847/1718] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:33.832610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f15d5-dd68-4274-9edb-8fa1aa4278da admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:33.842280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b416fd-3073-4a3c-ad0a-f2b5cdc6441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:33.843412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b416fd-3073-4a3c-ad0a-f2b5cdc6441d admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.843894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b416fd-3073-4a3c-ad0a-f2b5cdc6441d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.844384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b416fd-3073-4a3c-ad0a-f2b5cdc6441d admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.845668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b416fd-3073-4a3c-ad0a-f2b5cdc6441d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:33.849511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b416fd-3073-4a3c-ad0a-f2b5cdc6441d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:33.868499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 872/1719] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:33.874893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:33.886946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb06e1-5663-4f5d-89f0-5d4890965a15 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:33.895563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:33.896674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:33.897241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:33.897806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] PATH_INFO: `/v3/projects/5772603495734e4e8b9d5a442def0d3c/users/ebcac1ff19dd485682966e0cd43ddad8/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:33.916953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5772603495734e4e8b9d5a442def0d3c, user_id=ebcac1ff19dd485682966e0cd43ddad8, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:33.920564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:34.054762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 848/1720] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:33 2022] PUT /identity/v3/projects/5772603495734e4e8b9d5a442def0d3c/users/ebcac1ff19dd485682966e0cd43ddad8/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:34.066918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:34.067209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:34.067489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:34.098739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] MFA Rules not processed for user `ebcac1ff19dd485682966e0cd43ddad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:34.100118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:34.174594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 873/1721] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:34.179530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:34.250057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99d708-a769-46be-b3c0-848e94250c75 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:34.255284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:34.256140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:34.256602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:34.258775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:34.258775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:34.260368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:34.451183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 849/1722] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:34 2022] POST /identity/v3/projects => generated 387 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:34.454740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:34.470635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf227f66-cab1-4310-9011-8662274186e1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:34.544165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:34.545528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:34.545744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:34.546220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:34.547193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:34.550379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:34.569194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:34.695861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 874/1723] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:34 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:34.699695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:34.714236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd79060-53ea-4ebc-9a47-29fddc96e31f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:34.725509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:34.726579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:34.727256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:34.728854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:34.729760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:34.731742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:34.750363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 850/1724] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:34.754892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:34.765349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d709d-241e-4682-a3ad-ae402e760d6b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:34.770034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd103b7-5fee-4331-a253-f8b8cef42e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:34.770644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd103b7-5fee-4331-a253-f8b8cef42e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:34.770919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd103b7-5fee-4331-a253-f8b8cef42e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:34.771381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd103b7-5fee-4331-a253-f8b8cef42e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:34.772229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd103b7-5fee-4331-a253-f8b8cef42e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:34.773905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd103b7-5fee-4331-a253-f8b8cef42e23 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:34.784458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 875/1725] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:34.789528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:34.801197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d48a27-ba5b-481d-a29b-b6cb0c8082d6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:34.806179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:34.807023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:34.807357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:34.807562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] PATH_INFO: `/v3/projects/2fa75751f2bb445cb6efaa53c1990e89/users/759a91f220754317b0f9453dfd659392/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:34.831408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa75751f2bb445cb6efaa53c1990e89, user_id=759a91f220754317b0f9453dfd659392, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:34.834710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:34.996298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 851/1726] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:34 2022] PUT /identity/v3/projects/2fa75751f2bb445cb6efaa53c1990e89/users/759a91f220754317b0f9453dfd659392/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:35.008505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ddb4a-cdb7-4077-be45-21814dd7c8f0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:35.008962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ddb4a-cdb7-4077-be45-21814dd7c8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:35.009338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ddb4a-cdb7-4077-be45-21814dd7c8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:35.119719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a64ddb4a-cdb7-4077-be45-21814dd7c8f0 None None] MFA Rules not processed for user `759a91f220754317b0f9453dfd659392`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:35.121075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a64ddb4a-cdb7-4077-be45-21814dd7c8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:35.272437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 876/1727] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:35.277829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:35.451762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11cfa6-01b1-45d9-b9c0-43a02e07b2a3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:35.478240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:35.478240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:35.478240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:35.481638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:35.483035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:35.483746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:35.494913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:35.494913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:35.494913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:35.554239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:35.555909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:35.683611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:35.702070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 852/1728] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:35 2022] POST /identity/v3/projects => generated 387 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:35.708702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:35.738957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 877/1729] 10.222.0.45 () {58 vars in 1015 bytes} [Tue Oct 4 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:35.944134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d757a9-200e-4f16-9442-beb96c20bcc2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:35.957191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:35.957935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:35.958383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:35.958696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:35.959481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:35.964364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:36.485947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 853/1730] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:35 2022] POST /identity/v3/users => generated 300 bytes in 778 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:36.489912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:36.513083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1b14d-0bce-465c-bda7-80966bcf8594 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:36.526066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:36.552516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:36.552516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:36.552516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:36.552516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:36.552516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:36.552516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 878/1731] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:36.553506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:36.586218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e803099-ad9d-42bd-ad4e-44a8dc908090 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:36.593046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:36.595072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:36.595072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:36.595072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] PATH_INFO: `/v3/projects/f881781ff7554688a6837f8b9fbaa40c/users/271d1c82f9d0419cbb0434efc2a425a5/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:36.632489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f881781ff7554688a6837f8b9fbaa40c, user_id=271d1c82f9d0419cbb0434efc2a425a5, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:36.636254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:36.777274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 854/1732] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:36 2022] PUT /identity/v3/projects/f881781ff7554688a6837f8b9fbaa40c/users/271d1c82f9d0419cbb0434efc2a425a5/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:36.778087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:36.798685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316984b9-12d4-4819-b8e3-a6cf4957a603 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:36.836403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:36.838568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:37.015324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:37.016246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:37.016687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:37.017150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:37.017958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:37.019747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:37.064124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 879/1733] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:37.064124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:37.084833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4541d-7890-415d-835e-a23892acf51f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:37.107887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff4291-db1e-4ba1-a608-c3092ef919ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:37.108343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89685eb9-c6df-4939-8c72-46998b01a87a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:37.109094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4291-db1e-4ba1-a608-c3092ef919ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:37.109544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4291-db1e-4ba1-a608-c3092ef919ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:37.110022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff4291-db1e-4ba1-a608-c3092ef919ba admin admin] PATH_INFO: `/v3/projects/f881781ff7554688a6837f8b9fbaa40c/users/271d1c82f9d0419cbb0434efc2a425a5/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:37.115711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4291-db1e-4ba1-a608-c3092ef919ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f881781ff7554688a6837f8b9fbaa40c, user_id=271d1c82f9d0419cbb0434efc2a425a5, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:37.118697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff4291-db1e-4ba1-a608-c3092ef919ba admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:37.121275 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:37.121929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:37.123643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:37.127189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:37.148673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 880/1734] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:37 2022] PUT /identity/v3/projects/f881781ff7554688a6837f8b9fbaa40c/users/271d1c82f9d0419cbb0434efc2a425a5/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:37.167375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:37.167375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:37.167375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:37.167683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:37.169294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:37.229243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 855/1735] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:37.236810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] MFA Rules not processed for user `271d1c82f9d0419cbb0434efc2a425a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:37.239380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:37.452843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 881/1736] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:37.469630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:37.476649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:37.738680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb124e-084f-42b8-92ba-994962766266 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:37.961347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c778ec-5a41-442d-b827-4606bf3c0872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:37.968560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c778ec-5a41-442d-b827-4606bf3c0872 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:37.969071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c778ec-5a41-442d-b827-4606bf3c0872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:37.970421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c778ec-5a41-442d-b827-4606bf3c0872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:37.989253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:38.005182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:38.017614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c778ec-5a41-442d-b827-4606bf3c0872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:38.020165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c778ec-5a41-442d-b827-4606bf3c0872 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:38.057968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 856/1737] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:38.136830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:38.267382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e5f38-7fba-4cd6-8039-093abb627e89 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:38.279892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:38.281402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:38.282240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:38.283076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] PATH_INFO: `/v3/users/8490e6187a3344deb1f84fdb3c47b117` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:38.285725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8490e6187a3344deb1f84fdb3c47b117)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:38.290200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:38.386344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:38.627367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8490e6187a3344deb1f84fdb3c47b117'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:38.630608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8490e6187a3344deb1f84fdb3c47b117'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:38.648936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8490e6187a3344deb1f84fdb3c47b117'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:38.660890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8490e6187a3344deb1f84fdb3c47b117'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:38.751288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8490e6187a3344deb1f84fdb3c47b117'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:38.792054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 882/1738] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:37 2022] DELETE /identity/v3/users/8490e6187a3344deb1f84fdb3c47b117 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:38.796694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:38.796694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:38.797234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:38.865846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:38.867155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:39.046694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 857/1739] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:39.083556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:39.086837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:39.277520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc133-5e3a-4e7b-b011-e1c8836414c8 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:39.309482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:39.310579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:39.311072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:39.312271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:39.331929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:39.337403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:39.348054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 883/1740] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:40.177718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:40.182455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:40.311416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c799c94-7632-49ae-ab91-d8f395b7cb78 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:40.317818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:40.318695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:40.319306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:40.320135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:40.324563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:40.433765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:40.440707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:40.448767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 858/1741] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:40.610949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:40.612112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:40.658542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c375f-f360-4a4d-8bcc-56cf397aa7ac service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:40.670199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:40.670199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:40.688063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:40.690784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:40.696582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:40.713569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] PATH_INFO: `/v3/users/25500227a1914a51a4d50dd58e6f978d` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:40.719171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25500227a1914a51a4d50dd58e6f978d)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:40.721627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98a617-145f-4574-8c80-823590857d0f service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:40.725687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:40.731169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:40.738157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:40.738704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:40.739058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] PATH_INFO: `/v3/projects/b49452baafa34ffc925f15234fd56e23` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:40.740775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b49452baafa34ffc925f15234fd56e23)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:40.744136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:41.143939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b49452baafa34ffc925f15234fd56e23', 'request_id': 'req-0acaaa01-aeb1-4ba1-857f-415a64f4e412'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.181018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25500227a1914a51a4d50dd58e6f978d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.184774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25500227a1914a51a4d50dd58e6f978d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.211634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b49452baafa34ffc925f15234fd56e23', 'request_id': 'req-0acaaa01-aeb1-4ba1-857f-415a64f4e412'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.213552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25500227a1914a51a4d50dd58e6f978d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.221043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25500227a1914a51a4d50dd58e6f978d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.244702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b49452baafa34ffc925f15234fd56e23', 'request_id': 'req-0acaaa01-aeb1-4ba1-857f-415a64f4e412'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.273609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:41.299690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-945e309e-4d5d-43a7-9694-af27182bd019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25500227a1914a51a4d50dd58e6f978d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.337971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 884/1742] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:40 2022] DELETE /identity/v3/users/25500227a1914a51a4d50dd58e6f978d => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:41.339717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed736a71-23bf-413d-bde0-0a7f3aa596fe None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:41.340224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed736a71-23bf-413d-bde0-0a7f3aa596fe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:41.340828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed736a71-23bf-413d-bde0-0a7f3aa596fe None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:41.343564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 885/1743] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:35:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:41.345302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d60be-181c-433b-ba67-bfba1b1f3d8e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:41.346466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d60be-181c-433b-ba67-bfba1b1f3d8e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:41.350042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d60be-181c-433b-ba67-bfba1b1f3d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:41.384306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7c5d60be-181c-433b-ba67-bfba1b1f3d8e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:41.385755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7c5d60be-181c-433b-ba67-bfba1b1f3d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:41.411219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] The token cache is being invalidate because project b49452baafa34ffc925f15234fd56e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:41.411522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.412208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0acaaa01-aeb1-4ba1-857f-415a64f4e412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:41.426215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 859/1744] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:40 2022] DELETE /identity/v3/projects/b49452baafa34ffc925f15234fd56e23 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:41.427755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf911a-19b5-4129-8347-0adb0072971e None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:41.433850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf911a-19b5-4129-8347-0adb0072971e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:41.434275 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf911a-19b5-4129-8347-0adb0072971e None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:41.439194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 860/1745] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:35:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:41.439194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf911a-19b5-4129-8347-0adb0072971e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:41.441764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4caf911a-19b5-4129-8347-0adb0072971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:41.506530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 886/1746] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:41.520085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:41.536914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:41.537199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:41.537554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:41.591680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:41.599086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:41.617592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf911a-19b5-4129-8347-0adb0072971e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:41.622185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:41.624232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:41.624705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:41.625045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] PATH_INFO: `/v3/users/1ae8ad2ad4ef42bdbed07180ae3c0428` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:41.626972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ae8ad2ad4ef42bdbed07180ae3c0428)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:41.629086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:41.767653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 887/1747] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:35:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:41.768355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:41.770359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:41.895249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806d7bd-af10-4abd-b336-452d96aa21b8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:41.900056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469889a-00e5-480e-be87-351f73d89d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:41.901324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469889a-00e5-480e-be87-351f73d89d3e service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:41.901806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469889a-00e5-480e-be87-351f73d89d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:41.902274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469889a-00e5-480e-be87-351f73d89d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:41.919895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469889a-00e5-480e-be87-351f73d89d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:41.960672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469889a-00e5-480e-be87-351f73d89d3e service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:42.010596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 888/1748] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:42.011943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:42.012263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:42.012557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:42.023492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ae8ad2ad4ef42bdbed07180ae3c0428'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:42.025523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ae8ad2ad4ef42bdbed07180ae3c0428'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:42.038029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ae8ad2ad4ef42bdbed07180ae3c0428'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:42.039370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ae8ad2ad4ef42bdbed07180ae3c0428'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:42.068466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:42.070388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:42.092340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 889/1749] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:42.093361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:42.095219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:42.123090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d25aa-b95b-447e-88cd-824ce8171711 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:42.132328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:42.134236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ae8ad2ad4ef42bdbed07180ae3c0428'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:42.134385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:42.135035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:42.135361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:42.138915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:42.158845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:42.160719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:42.161757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 861/1750] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:41 2022] DELETE /identity/v3/users/1ae8ad2ad4ef42bdbed07180ae3c0428 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:42.166348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:42.167888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:42.203044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 890/1751] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:42.383801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c4f7-8b96-4ce0-817c-05096b850bef admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:42.424909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:42.430427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:42.430427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:42.430427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:42.460700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:42.463023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:42.546876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 862/1752] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:43.991668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:44.309765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:44.355314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.359172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:44.422031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef60d4-eb90-4397-9eff-0c078d8bb470 service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.480935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.484268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:44.637817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.639532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.640491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.641470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] PATH_INFO: `/v3/projects/696674217f3243c79f51f3a1639a3e95` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.644494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696674217f3243c79f51f3a1639a3e95)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.646976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.729799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e26b7-77b7-422d-b02f-b159a78a3654 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.748835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.749613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.749902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.750179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] PATH_INFO: `/v3/projects/553cbf99aa0e4957ae876da56ff6f45e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.766737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=553cbf99aa0e4957ae876da56ff6f45e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.768761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.865812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696674217f3243c79f51f3a1639a3e95', 'request_id': 'req-b07251cb-32ae-499b-82b3-6a20816074d9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:44.889607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696674217f3243c79f51f3a1639a3e95', 'request_id': 'req-b07251cb-32ae-499b-82b3-6a20816074d9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:44.969551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553cbf99aa0e4957ae876da56ff6f45e', 'request_id': 'req-f8b2794a-19cb-4999-9d84-26b6fc817f2d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:44.980277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553cbf99aa0e4957ae876da56ff6f45e', 'request_id': 'req-f8b2794a-19cb-4999-9d84-26b6fc817f2d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:44.991892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553cbf99aa0e4957ae876da56ff6f45e', 'request_id': 'req-f8b2794a-19cb-4999-9d84-26b6fc817f2d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.003033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696674217f3243c79f51f3a1639a3e95', 'request_id': 'req-b07251cb-32ae-499b-82b3-6a20816074d9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.015995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] The token cache is being invalidate because project 696674217f3243c79f51f3a1639a3e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:45.016448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.017315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b07251cb-32ae-499b-82b3-6a20816074d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.019564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 891/1753] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:44 2022] DELETE /identity/v3/projects/696674217f3243c79f51f3a1639a3e95 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:45.024915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185328-80a4-4b43-bfdb-9d2e59e90f4f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:45.025205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185328-80a4-4b43-bfdb-9d2e59e90f4f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:45.025597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7185328-80a4-4b43-bfdb-9d2e59e90f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:45.049772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a7185328-80a4-4b43-bfdb-9d2e59e90f4f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:45.051391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a7185328-80a4-4b43-bfdb-9d2e59e90f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:45.181809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] The token cache is being invalidate because project 553cbf99aa0e4957ae876da56ff6f45e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:45.182107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.186610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.186610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 863/1754] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:44 2022] DELETE /identity/v3/projects/553cbf99aa0e4957ae876da56ff6f45e => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:45.190974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:45.192688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:45.198864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 892/1755] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:45.209223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:45.209223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:45.209223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:45.213205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2794a-19cb-4999-9d84-26b6fc817f2d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:45.219304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:45.222066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:45.222365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:45.222647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] PATH_INFO: `/v3/users/0b74b4dfa5de45579f047c02501438e4` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:45.224839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b74b4dfa5de45579f047c02501438e4)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:45.227556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:45.244572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:45.246072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:45.286818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 893/1756] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:45.301662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:45.349834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ecf73-12f1-4e37-9d9c-2b84f1d7c4da None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:45.370072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:45.375930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:45.375930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:45.375930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:45.375930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:45.376781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:45.380420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:45.421079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 894/1757] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:45.422337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:45.437001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483bb1b-40ab-44ed-857d-bcd389261798 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:45.444926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:45.446129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:45.446564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:45.447512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:45.448427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:45.452185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:45.559340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b74b4dfa5de45579f047c02501438e4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.560368 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b74b4dfa5de45579f047c02501438e4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.579382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b74b4dfa5de45579f047c02501438e4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.580131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b74b4dfa5de45579f047c02501438e4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.656975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 895/1758] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:45 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:45.656975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:45.662093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:45.711609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b74b4dfa5de45579f047c02501438e4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:45.770587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 864/1759] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:45 2022] DELETE /identity/v3/users/0b74b4dfa5de45579f047c02501438e4 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:45.855185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b19f-7fab-495c-aa23-1ced1c3fb42a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.033406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.034156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.034337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.034669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.035328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.037487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.205304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 896/1760] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:45 2022] POST /identity/v3/users => generated 290 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:46.210436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.233580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd811d6-672a-4588-8430-f6215468336e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.247132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.248685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.249000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.249275 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.253123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.255414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.280111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 865/1761] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:46.286170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.299719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24248d3-42e8-4380-9241-726aab5b7e29 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.307826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459fdfe-4741-48ec-a300-d8846eda8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.308679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459fdfe-4741-48ec-a300-d8846eda8ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.309135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459fdfe-4741-48ec-a300-d8846eda8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.311134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459fdfe-4741-48ec-a300-d8846eda8ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.312032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459fdfe-4741-48ec-a300-d8846eda8ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.321305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459fdfe-4741-48ec-a300-d8846eda8ddb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.357862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 897/1762] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:46.360630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.391312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f9bbe-0895-4240-a161-e0c9a146b351 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.404568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.407104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.407708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.407987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] PATH_INFO: `/v3/projects/ae892653dc214584b304432d2bda4a87/users/87c8cc6282d74f31876fa8d25cb0e4b1/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.445183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae892653dc214584b304432d2bda4a87, user_id=87c8cc6282d74f31876fa8d25cb0e4b1, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.449556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.594854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 866/1763] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:46 2022] PUT /identity/v3/projects/ae892653dc214584b304432d2bda4a87/users/87c8cc6282d74f31876fa8d25cb0e4b1/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:46.607766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.607766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.607766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.652983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] MFA Rules not processed for user `87c8cc6282d74f31876fa8d25cb0e4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:46.654735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:46.906069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 898/1764] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:46.910946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.095422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4720f-50e0-489a-b4e8-852d9360f9ed admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.110154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.110838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.111121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.111392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.112096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.114455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.202817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:47.272798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 867/1765] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:46 2022] POST /identity/v3/projects => generated 381 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:47.273682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.289504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916fa07b-ffce-47f8-b1f7-a2e43b4c8cd5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.374628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.377496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.455406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.457129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.457524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.458481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.460446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.465731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.553565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7156a6-84a3-4b9c-9686-edc8a9f89632 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.561097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.561097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.561097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.561097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.568760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.656369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 899/1766] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:47 2022] POST /identity/v3/users => generated 297 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:47.658121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.677452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d78439-f3e3-4e57-a1a7-8d7f59af04e6 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.684508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.687577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.687866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.688147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.689018 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.693902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.738561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 900/1767] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:47.739839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.765445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11315a6-0f4f-48e6-881e-27ec74c8e5fb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.778533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.781853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.782807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.784081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] PATH_INFO: `/v3/projects/216861a7b0b7408cb6b4340ba5bab9b9/users/6c67ce67d5794c469decb917c8f29aa6/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.789962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.796650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.834130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216861a7b0b7408cb6b4340ba5bab9b9, user_id=6c67ce67d5794c469decb917c8f29aa6, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.835475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 868/1768] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:47.837776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:48.234203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 901/1769] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:47 2022] PUT /identity/v3/projects/216861a7b0b7408cb6b4340ba5bab9b9/users/6c67ce67d5794c469decb917c8f29aa6/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:48.238207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:48.253897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98c644-3afc-42aa-8c30-67b394b87d63 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:48.284479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:48.287134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:48.352561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:48.353927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.354379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.354972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.356165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:48.360508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:48.375429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 869/1770] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:48.378908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:48.393288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ba22-d4dc-4b1d-8137-227d8c9578ee admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:48.400929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74a3c4-c5fc-4353-99ca-cace7ae0103c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:48.402341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a3c4-c5fc-4353-99ca-cace7ae0103c admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.402341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a3c4-c5fc-4353-99ca-cace7ae0103c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.403214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74a3c4-c5fc-4353-99ca-cace7ae0103c admin admin] PATH_INFO: `/v3/projects/216861a7b0b7408cb6b4340ba5bab9b9/users/6c67ce67d5794c469decb917c8f29aa6/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.406536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a3c4-c5fc-4353-99ca-cace7ae0103c admin admin] RBAC: Authorizing `identity:create_grant(project_id=216861a7b0b7408cb6b4340ba5bab9b9, user_id=6c67ce67d5794c469decb917c8f29aa6, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:48.407587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8775d98-6bd2-4db4-9062-cdc85be2d2e3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:48.424743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:48.425568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.426163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.426163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] PATH_INFO: `/v3/projects/6ac4979497254fc48e1bd7b47ff46b02` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.426834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74a3c4-c5fc-4353-99ca-cace7ae0103c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:48.431376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac4979497254fc48e1bd7b47ff46b02)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:48.435656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:48.460178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 870/1771] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:48 2022] PUT /identity/v3/projects/216861a7b0b7408cb6b4340ba5bab9b9/users/6c67ce67d5794c469decb917c8f29aa6/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:48.476394 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bd23c-faf5-4152-8d4e-80c48810e18a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.476688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bd23c-faf5-4152-8d4e-80c48810e18a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.477304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bd23c-faf5-4152-8d4e-80c48810e18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.506853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-060bd23c-faf5-4152-8d4e-80c48810e18a None None] MFA Rules not processed for user `6c67ce67d5794c469decb917c8f29aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:48.508153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-060bd23c-faf5-4152-8d4e-80c48810e18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:48.589570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 871/1772] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:48.628655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:48.650271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac4979497254fc48e1bd7b47ff46b02', 'request_id': 'req-613ab403-efb8-4416-b434-c185c565b53b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:48.656779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac4979497254fc48e1bd7b47ff46b02', 'request_id': 'req-613ab403-efb8-4416-b434-c185c565b53b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:48.739640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac4979497254fc48e1bd7b47ff46b02', 'request_id': 'req-613ab403-efb8-4416-b434-c185c565b53b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:48.753574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] The token cache is being invalidate because project 6ac4979497254fc48e1bd7b47ff46b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:48.753907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:48.754526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-613ab403-efb8-4416-b434-c185c565b53b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:48.756430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 902/1773] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:48 2022] DELETE /identity/v3/projects/6ac4979497254fc48e1bd7b47ff46b02 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:48.761333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.761626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.761923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.784413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:48.785675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:48.858124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 872/1774] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:48.871065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.871065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.871065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.891021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:48.893580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:48.902811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 903/1775] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:48.913022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:48.916574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46aaa-5e55-4653-a723-f02c2519aa20 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:48.927288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:48.927926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.928202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.928676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.929370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:48.929796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:48.931505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:48.945553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 873/1776] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:35:48.950010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:48.961234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1fa35-e9fc-4798-a613-7b2e22b8f0ba None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:48.965960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:48.966702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:48.967154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:48.967499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:48.968364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:48.971657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:49.078333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 904/1777] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:48 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:49.080939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:49.091464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761501b-cc74-441b-95de-b720792628b3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:49.168014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:49.168014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.168477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.169009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.171063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:49.173338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:49.321848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 874/1778] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:49 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:49.325572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:49.342747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eebaaf-2f02-46b7-ad23-e473a0f27e97 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:49.353177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:49.356062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.356062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.356062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.356062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:49.357262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:49.368577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 905/1779] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:49.373449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:49.386221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803febf-f4a8-4996-930c-50442d7d16b3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:49.394449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfce93-8a0d-4644-be42-01e133e3c32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:49.394449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfce93-8a0d-4644-be42-01e133e3c32f admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.394449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfce93-8a0d-4644-be42-01e133e3c32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.394449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfce93-8a0d-4644-be42-01e133e3c32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.394801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfce93-8a0d-4644-be42-01e133e3c32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:49.396510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfce93-8a0d-4644-be42-01e133e3c32f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:49.410811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 875/1780] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:49.415054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:49.426014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059822e-4aa5-40c8-ae3e-2289e8d2758c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:49.430653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:49.431307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.431590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.431867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] PATH_INFO: `/v3/projects/5e5f7bf8ae0241aba0505d22e1f19c3e/users/a540134116144d21a036c2c0a74897cc/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.453796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e5f7bf8ae0241aba0505d22e1f19c3e, user_id=a540134116144d21a036c2c0a74897cc, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:49.456043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:49.574433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 906/1781] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:49 2022] PUT /identity/v3/projects/5e5f7bf8ae0241aba0505d22e1f19c3e/users/a540134116144d21a036c2c0a74897cc/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:49.585095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.585381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.585657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.617715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] MFA Rules not processed for user `a540134116144d21a036c2c0a74897cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:49.619844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:49.688119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 876/1782] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:49.692630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:49.759111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5b77f-17fb-4497-9cac-28b5df8fc5a1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:49.765003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:49.765618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.765897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.766276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.766993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:49.772654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:49.855515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 907/1783] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:49 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:49.861042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:49.873154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca51b0-dbdd-402c-8601-b02c55af6f50 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:49.941992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:49.942566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:49.942974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:49.943218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:49.943993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:49.946840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.054588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 877/1784] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:49 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:50.058677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:50.071070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fa066-cfb2-483b-a4a0-3f2e79cedcf7 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:50.076691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:50.077401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.077633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.077911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.078556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:50.080170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.101127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 908/1785] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:50.104154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:50.114577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba02956-3d0d-4714-94d6-8f51583944c9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:50.119271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:50.119927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.120210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.120487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] PATH_INFO: `/v3/projects/02417a504b934caf9e1b2b8e7e903a1a/users/528e418764e54056a4b03d9bdf506b02/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.135002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02417a504b934caf9e1b2b8e7e903a1a, user_id=528e418764e54056a4b03d9bdf506b02, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:50.137692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.196764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:50.238338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 878/1786] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:50 2022] PUT /identity/v3/projects/02417a504b934caf9e1b2b8e7e903a1a/users/528e418764e54056a4b03d9bdf506b02/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:50.242355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:50.254483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e41ca-6358-4d74-8fbe-a0d835948fbf admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:50.317859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f35c0-8978-470e-bf1e-64f2bc45e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:50.318475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f35c0-8978-470e-bf1e-64f2bc45e3ce admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.318758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f35c0-8978-470e-bf1e-64f2bc45e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.319255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f35c0-8978-470e-bf1e-64f2bc45e3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.319891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f35c0-8978-470e-bf1e-64f2bc45e3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:50.321637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f35c0-8978-470e-bf1e-64f2bc45e3ce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.334493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 909/1787] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:50.344923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:50.355344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da4b6c-6c28-4507-88e3-eefa104ea1e4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:50.360710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:50.361782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.362212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.362662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] PATH_INFO: `/v3/projects/02417a504b934caf9e1b2b8e7e903a1a/users/528e418764e54056a4b03d9bdf506b02/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.364834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02417a504b934caf9e1b2b8e7e903a1a, user_id=528e418764e54056a4b03d9bdf506b02, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:50.370396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.391620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 879/1788] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:50 2022] PUT /identity/v3/projects/02417a504b934caf9e1b2b8e7e903a1a/users/528e418764e54056a4b03d9bdf506b02/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:50.402287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.402792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.403653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.432583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] MFA Rules not processed for user `528e418764e54056a4b03d9bdf506b02`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:50.434015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:50.512423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 910/1789] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:50.521808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:50.523568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:50.588233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6efcf0-fcab-4db4-a302-8f526377d474 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:50.592432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:50.593088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.593426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.593702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.603809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:50.605360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.618369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 880/1790] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:50.700549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:50.764225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:50.765717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:50.777760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0432c7f-24a1-4375-962a-26a634bd707f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:50.781826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:50.782431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:50.782709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:50.782983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:50.841760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:50.843329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:50.861006 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 911/1791] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:52.379879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:52.997149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:53.064809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:53.073793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12678b-6ae0-4e6e-9ae0-2f316d46e492 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:53.079751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:53.087487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:53.087487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:53.087487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:53.087487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:53.158289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:53.159833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bed5d-66c3-485c-9da7-176bb812935f service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:53.172337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 881/1792] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:53.229973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:53.231429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:53.308182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b985a31-292e-43f8-b674-4b9c77848fbb service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:53.314369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:53.315028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:53.315329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:53.315631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] PATH_INFO: `/v3/users/ebcac1ff19dd485682966e0cd43ddad8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:53.317080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcac1ff19dd485682966e0cd43ddad8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:53.319132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:54.452694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcac1ff19dd485682966e0cd43ddad8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:54.453834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcac1ff19dd485682966e0cd43ddad8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:54.462242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcac1ff19dd485682966e0cd43ddad8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:54.463016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcac1ff19dd485682966e0cd43ddad8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:54.551230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-89e278a5-c00c-4ba4-839f-07618dc83cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcac1ff19dd485682966e0cd43ddad8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:54.602215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 912/1793] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:53 2022] DELETE /identity/v3/users/ebcac1ff19dd485682966e0cd43ddad8 => generated 0 bytes in 1373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:54.606966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b3c2a-f3f0-42cd-b841-f3e83ebf18ad None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:54.607833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b3c2a-f3f0-42cd-b841-f3e83ebf18ad None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:54.607833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b3c2a-f3f0-42cd-b841-f3e83ebf18ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:54.642372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-172b3c2a-f3f0-42cd-b841-f3e83ebf18ad None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:54.643889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-172b3c2a-f3f0-42cd-b841-f3e83ebf18ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:54.763993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 882/1794] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:54.771978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988319c-cf3b-4fe2-ae53-17be5e7a0998 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:54.771978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988319c-cf3b-4fe2-ae53-17be5e7a0998 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:54.772332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988319c-cf3b-4fe2-ae53-17be5e7a0998 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:54.773894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 913/1795] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:35:54.801285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:54.801285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:54.801551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988319c-cf3b-4fe2-ae53-17be5e7a0998 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:54.801551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7988319c-cf3b-4fe2-ae53-17be5e7a0998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:54.803944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:54.861913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:54.861913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:54.894748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988319c-cf3b-4fe2-ae53-17be5e7a0998 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:54.899330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:54.900086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:54.900460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:54.900833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:54.912380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:54.914978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:54.930513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 914/1796] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:55.009625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 883/1797] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:35:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:55.089152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.091841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:55.124850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db745a59-5cb8-4821-a158-e39451df4912 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.141639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.142268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.142555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.142810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.160439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.161356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:55.174967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 915/1798] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:55.407670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.409163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:55.475291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed5aaf-fb9c-4331-b374-d9c40f246cc6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.479844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c67ce67d5794c469decb917c8f29aa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '216861a7b0b7408cb6b4340ba5bab9b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.480490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.480774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.481150 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.482404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:55.482895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.484284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:55.495871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 884/1799] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:55.500380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.507661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.509742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:55.514662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb43701-fca5-4851-88a5-6309f19b3a9f service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.527243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c67ce67d5794c469decb917c8f29aa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '216861a7b0b7408cb6b4340ba5bab9b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.528202 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.529381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.530001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.530608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8329e-fd12-4a15-845f-66919b20bc99 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.531420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.534805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:55.537361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.538021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.538543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.539024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] PATH_INFO: `/v3/projects/5772603495734e4e8b9d5a442def0d3c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.542226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5772603495734e4e8b9d5a442def0d3c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.546918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:55.632843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 916/1800] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:55 2022] POST /identity/v3/projects => generated 355 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:55.637340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.688145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5772603495734e4e8b9d5a442def0d3c', 'request_id': 'req-e5921eae-ef94-4031-9d09-cf580413160d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:55.697324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5772603495734e4e8b9d5a442def0d3c', 'request_id': 'req-e5921eae-ef94-4031-9d09-cf580413160d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:55.711693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5772603495734e4e8b9d5a442def0d3c', 'request_id': 'req-e5921eae-ef94-4031-9d09-cf580413160d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:55.739392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815a7e5-b154-42ee-b25b-d57b2a07a242 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.796674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c67ce67d5794c469decb917c8f29aa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '216861a7b0b7408cb6b4340ba5bab9b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.797320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.797601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.797878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.798664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:55.798970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.800610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:55.814277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 917/1801] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:55.822890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] The token cache is being invalidate because project 5772603495734e4e8b9d5a442def0d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:55.823190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:55.833145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:55.835803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 885/1802] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:55 2022] DELETE /identity/v3/projects/5772603495734e4e8b9d5a442def0d3c => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:55.839823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.842381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:55.877198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:55.878943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:55.904148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5921eae-ef94-4031-9d09-cf580413160d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.916009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c67ce67d5794c469decb917c8f29aa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '216861a7b0b7408cb6b4340ba5bab9b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.916687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.917055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.917338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] PATH_INFO: `/v3/projects/d88c93b710fd4f70bda8b6a48787403e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.919893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d88c93b710fd4f70bda8b6a48787403e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.923159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:55.933417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7787b1-5a81-440b-8ee4-7cd3ed40240c tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:55.939769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:55.940466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:55.940747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:55.941054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] PATH_INFO: `/v3/users/759a91f220754317b0f9453dfd659392` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:55.942339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=759a91f220754317b0f9453dfd659392)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:55.944202 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:56.224203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88c93b710fd4f70bda8b6a48787403e', 'request_id': 'req-876b688f-bb01-4b38-80bd-083ba366e450'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.231669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759a91f220754317b0f9453dfd659392'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.233084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759a91f220754317b0f9453dfd659392'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.238391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:56.238856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88c93b710fd4f70bda8b6a48787403e', 'request_id': 'req-876b688f-bb01-4b38-80bd-083ba366e450'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.246433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88c93b710fd4f70bda8b6a48787403e', 'request_id': 'req-876b688f-bb01-4b38-80bd-083ba366e450'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.258374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759a91f220754317b0f9453dfd659392'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.258374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759a91f220754317b0f9453dfd659392'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.357270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759a91f220754317b0f9453dfd659392'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.389787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 918/1803] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:55 2022] DELETE /identity/v3/users/759a91f220754317b0f9453dfd659392 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:56.448164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:56.489435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] The token cache is being invalidate because project d88c93b710fd4f70bda8b6a48787403e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:56.489725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.492550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:56.497805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 886/1804] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:55 2022] DELETE /identity/v3/projects/d88c93b710fd4f70bda8b6a48787403e => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:56.959581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:56.961079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:57.028078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb1b2f-9a87-4fc0-9f62-f5057f6394d5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:57.033414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:57.034069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:57.034346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:57.034684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] PATH_INFO: `/v3/projects/2fa75751f2bb445cb6efaa53c1990e89` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:57.035801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa75751f2bb445cb6efaa53c1990e89)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:57.037775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:57.166663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa75751f2bb445cb6efaa53c1990e89', 'request_id': 'req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.174670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa75751f2bb445cb6efaa53c1990e89', 'request_id': 'req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.274821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:57.275892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa75751f2bb445cb6efaa53c1990e89', 'request_id': 'req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.276694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:57.304452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] The token cache is being invalidate because project 2fa75751f2bb445cb6efaa53c1990e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:57.308748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.308748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.308748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 919/1805] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:56 2022] DELETE /identity/v3/projects/2fa75751f2bb445cb6efaa53c1990e89 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:57.314209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:57.315911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:57.372582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b688f-bb01-4b38-80bd-083ba366e450 tempest-VolumeQuotaClassesTest-1373931882 tempest-VolumeQuotaClassesTest-1373931882-project-admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:57.454445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927e0d5-a4d2-4f9e-b7f9-e837addcbed1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:57.459992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:57.460839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:57.461304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:57.461730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] PATH_INFO: `/v3/users/271d1c82f9d0419cbb0434efc2a425a5` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:57.462500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:57.463041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=271d1c82f9d0419cbb0434efc2a425a5)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:57.463563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:57.464010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:57.464470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] PATH_INFO: `/v3/users/87c8cc6282d74f31876fa8d25cb0e4b1` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:57.465060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:57.466790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c8cc6282d74f31876fa8d25cb0e4b1)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:57.468952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:57.792667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '271d1c82f9d0419cbb0434efc2a425a5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.793731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '271d1c82f9d0419cbb0434efc2a425a5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.795121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c8cc6282d74f31876fa8d25cb0e4b1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.796070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c8cc6282d74f31876fa8d25cb0e4b1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.803563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '271d1c82f9d0419cbb0434efc2a425a5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.804393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '271d1c82f9d0419cbb0434efc2a425a5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.804536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c8cc6282d74f31876fa8d25cb0e4b1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.805865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c8cc6282d74f31876fa8d25cb0e4b1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.904185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '271d1c82f9d0419cbb0434efc2a425a5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:57.928174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 920/1806] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:57 2022] DELETE /identity/v3/users/271d1c82f9d0419cbb0434efc2a425a5 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:57.929140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:57.930519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:57.938616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:58.029954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e57611-c8ba-4976-b702-3f140906b374 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:58.035257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:58.036484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:58.036888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:58.037355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] PATH_INFO: `/v3/users/a540134116144d21a036c2c0a74897cc` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:58.038586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a540134116144d21a036c2c0a74897cc)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:58.040616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:58.048610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ec3ef5a0-8ca2-4969-81df-02c3752073f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c8cc6282d74f31876fa8d25cb0e4b1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:58.060208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:58.085897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 887/1807] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:57 2022] DELETE /identity/v3/users/87c8cc6282d74f31876fa8d25cb0e4b1 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:58.090534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:58.095698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:58.095698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:58.140447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:58.141906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:58.227948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 888/1808] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:58.239300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:58.242412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:58.323696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a540134116144d21a036c2c0a74897cc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:58.324513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a540134116144d21a036c2c0a74897cc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:58.341093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a540134116144d21a036c2c0a74897cc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:58.342371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a540134116144d21a036c2c0a74897cc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:58.368789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c71280-e2ae-4759-a93c-e45e7dde46c4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:58.380633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:58.385313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:58.385313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:58.385313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:58.385313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:58.398350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:58.400007 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41991c7d-fbec-4057-81a7-2b71639ed094 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:58.419377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 889/1809] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:35:58.448273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a540134116144d21a036c2c0a74897cc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:58.495739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 921/1810] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:57 2022] DELETE /identity/v3/users/a540134116144d21a036c2c0a74897cc => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:58.506019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:58.506666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:58.508029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:58.540051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:58.540051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:58.713912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:58.720226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:58.747808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 890/1811] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:58.773127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:58.783283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:58.915130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015b6ae-030b-4394-b410-00f8db8b8391 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:58.927751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb2d8e-d26e-483b-aed4-49b065c50f54 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:58.928352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0228-b9a9-4b83-9844-9dfcaacc1d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:58.928986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0228-b9a9-4b83-9844-9dfcaacc1d17 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:58.929273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0228-b9a9-4b83-9844-9dfcaacc1d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:58.929602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0228-b9a9-4b83-9844-9dfcaacc1d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:58.941379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:58.942133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:58.942242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:58.942512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] PATH_INFO: `/v3/projects/f881781ff7554688a6837f8b9fbaa40c` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:58.943715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f881781ff7554688a6837f8b9fbaa40c)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:58.945690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:58.964555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0228-b9a9-4b83-9844-9dfcaacc1d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:58.966971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0228-b9a9-4b83-9844-9dfcaacc1d17 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:58.991359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 891/1812] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:59.136322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f881781ff7554688a6837f8b9fbaa40c', 'request_id': 'req-92c227af-8754-4aab-b586-3a930c5816dd'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.146335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f881781ff7554688a6837f8b9fbaa40c', 'request_id': 'req-92c227af-8754-4aab-b586-3a930c5816dd'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.218724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f881781ff7554688a6837f8b9fbaa40c', 'request_id': 'req-92c227af-8754-4aab-b586-3a930c5816dd'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.236176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] The token cache is being invalidate because project f881781ff7554688a6837f8b9fbaa40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:59.236472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.237210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-92c227af-8754-4aab-b586-3a930c5816dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.242250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 922/1813] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:58 2022] DELETE /identity/v3/projects/f881781ff7554688a6837f8b9fbaa40c => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:59.248818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.249348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.249743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.272662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:59.274027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:59.343643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 892/1814] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:35:59.352117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.352384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.352661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.374370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:59.375631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:59.393099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 923/1815] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:59.394494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:59.397058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:59.399098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:59.410980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fbf36-7aa9-4fcc-8a94-83c3feba46ce None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:59.418136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba517ae-1341-4c01-b0fc-09353178e970 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:59.423576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:59.425316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.425899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.432053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.432053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:59.432053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:59.432053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:59.432368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:59.432368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.432368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.432368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] PATH_INFO: `/v3/projects/ae892653dc214584b304432d2bda4a87` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.433583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae892653dc214584b304432d2bda4a87)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:59.438308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:59.472830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 893/1816] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:59.477583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:59.489632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d051b4-88d1-46a4-ab30-47b9dc805f1b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:59.495383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:59.496287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.496498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.496755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.497479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:59.499553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:59.577686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae892653dc214584b304432d2bda4a87', 'request_id': 'req-913c20ae-0199-4340-9280-8ab079469a23'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.584803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae892653dc214584b304432d2bda4a87', 'request_id': 'req-913c20ae-0199-4340-9280-8ab079469a23'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.625520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 894/1817] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:59 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:59.629526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:59.700564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3e431-8c57-407f-a25e-b781c18bd4ce admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:59.705804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:59.707278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.707564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.707855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.708760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:59.710859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:59.756640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae892653dc214584b304432d2bda4a87', 'request_id': 'req-913c20ae-0199-4340-9280-8ab079469a23'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.774296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] The token cache is being invalidate because project ae892653dc214584b304432d2bda4a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:59.774599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.775346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:59.777316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 924/1818] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:59 2022] DELETE /identity/v3/projects/ae892653dc214584b304432d2bda4a87 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:59.778979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:59.780500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:59.835493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 895/1819] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:59 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:59.836449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:59.838159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:59.851016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80f321-d1a3-45b0-a45f-6bd968158f3a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:59.853613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c20ae-0199-4340-9280-8ab079469a23 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:59.855832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:59.856497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.856788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.857097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] PATH_INFO: `/v3/users/6c67ce67d5794c469decb917c8f29aa6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.858538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c67ce67d5794c469decb917c8f29aa6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:59.859326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:59.860529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:59.860529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:59.860893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] PATH_INFO: `/v3/projects/5e5f7bf8ae0241aba0505d22e1f19c3e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:59.862126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:59.862465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e5f7bf8ae0241aba0505d22e1f19c3e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:59.864450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:00.019502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:00.080800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c67ce67d5794c469decb917c8f29aa6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.081814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c67ce67d5794c469decb917c8f29aa6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.081953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e5f7bf8ae0241aba0505d22e1f19c3e', 'request_id': 'req-5d422d17-e18c-4bb5-a682-495c2f73ef55'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.088513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e5f7bf8ae0241aba0505d22e1f19c3e', 'request_id': 'req-5d422d17-e18c-4bb5-a682-495c2f73ef55'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.091055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c67ce67d5794c469decb917c8f29aa6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.092113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c67ce67d5794c469decb917c8f29aa6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.161920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e5f7bf8ae0241aba0505d22e1f19c3e', 'request_id': 'req-5d422d17-e18c-4bb5-a682-495c2f73ef55'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.174728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] The token cache is being invalidate because project 5e5f7bf8ae0241aba0505d22e1f19c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:00.175028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.175689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.180100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 925/1820] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:59 2022] DELETE /identity/v3/projects/5e5f7bf8ae0241aba0505d22e1f19c3e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:00.180100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:00.180100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:00.237077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c67ce67d5794c469decb917c8f29aa6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.257928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 896/1821] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:59 2022] DELETE /identity/v3/users/6c67ce67d5794c469decb917c8f29aa6 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:00.261269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:00.261269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:00.267164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d422d17-e18c-4bb5-a682-495c2f73ef55 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:00.346792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216c8ff-0ef8-48e3-bc3a-406c603c1e5b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:00.350931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:00.352030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:00.354173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:00.354892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:00.355585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:00.360790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:00.360790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:00.360790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:00.360790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] PATH_INFO: `/v3/users/528e418764e54056a4b03d9bdf506b02` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:00.361160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:00.361246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=528e418764e54056a4b03d9bdf506b02)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:00.362591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:00.374095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 926/1822] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:00.378405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:00.389996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ad684-faa6-4c49-8604-631266b90cc3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:00.395596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:00.396355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:00.396667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:00.396972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:00.397674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:00.399306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:00.409984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 927/1823] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:00.415272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:00.425789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745fcc7-5bcc-4b6c-95e9-a361d5bc211b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:00.431276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015e95a-8281-4a3a-bd3d-014c3b5e01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:00.431995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e95a-8281-4a3a-bd3d-014c3b5e01d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:00.432273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e95a-8281-4a3a-bd3d-014c3b5e01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:00.432546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015e95a-8281-4a3a-bd3d-014c3b5e01d9 admin admin] PATH_INFO: `/v3/projects/1a62bf6be8d440edbf3e70d7cc897c41/users/8d7c0f34fbdc404593f84f48d9e4e868/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:00.452793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015e95a-8281-4a3a-bd3d-014c3b5e01d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a62bf6be8d440edbf3e70d7cc897c41, user_id=8d7c0f34fbdc404593f84f48d9e4e868, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:00.452793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015e95a-8281-4a3a-bd3d-014c3b5e01d9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:00.582277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 928/1824] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:00 2022] PUT /identity/v3/projects/1a62bf6be8d440edbf3e70d7cc897c41/users/8d7c0f34fbdc404593f84f48d9e4e868/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:00.592721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:00.593084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:00.593449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:00.638367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] MFA Rules not processed for user `8d7c0f34fbdc404593f84f48d9e4e868`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:00.639604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:00.660638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528e418764e54056a4b03d9bdf506b02'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.661869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528e418764e54056a4b03d9bdf506b02'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.671522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528e418764e54056a4b03d9bdf506b02'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.672343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528e418764e54056a4b03d9bdf506b02'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.726329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 929/1825] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:00.730658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:00.732724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:00.790455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528e418764e54056a4b03d9bdf506b02'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:00.812529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5ac35-42ae-4666-9302-d55650675c81 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:00.820983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:00.821680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:00.822372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:00.822685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:00.823485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:00.825889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:00.833859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:00.862643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 897/1826] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:00 2022] DELETE /identity/v3/users/528e418764e54056a4b03d9bdf506b02 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:00.968210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 930/1827] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:00 2022] POST /identity/v3/projects => generated 391 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:00.973235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.100789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce38483-3e88-453b-9c42-b2f50e6a66e0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.196515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.198446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.198741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.199730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.199730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.203770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:01.321384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 898/1828] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:00 2022] POST /identity/v3/users => generated 302 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:01.326172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.339257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13b840-a31c-478f-ad4f-75926f51901c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.344744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.348970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.348970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.348970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.348970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.349274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:01.380947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.396597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 931/1829] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:01.403477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b7174-f4a6-4aec-9a5b-665477f59b9a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.410117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.410805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.411085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.411361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] PATH_INFO: `/v3/projects/a3b465f566d34d86b4c6acb60f965995/users/8cf5e398b68542599bf5f474dcd0f2de/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.440852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b465f566d34d86b4c6acb60f965995, user_id=8cf5e398b68542599bf5f474dcd0f2de, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.440852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:01.443214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.445310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:01.462955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72d54f-0b23-4c7f-a380-105f436f8123 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.472291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.473629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.473919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.474191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] PATH_INFO: `/v3/projects/216861a7b0b7408cb6b4340ba5bab9b9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.475803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216861a7b0b7408cb6b4340ba5bab9b9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.478560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:01.548009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 899/1830] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:01 2022] PUT /identity/v3/projects/a3b465f566d34d86b4c6acb60f965995/users/8cf5e398b68542599bf5f474dcd0f2de/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:01.549083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.613378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216861a7b0b7408cb6b4340ba5bab9b9', 'request_id': 'req-2a87d96e-014a-4290-901e-5decbc917b74'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:01.617944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216861a7b0b7408cb6b4340ba5bab9b9', 'request_id': 'req-2a87d96e-014a-4290-901e-5decbc917b74'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:01.665859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42f67-6064-4e3e-afed-d342fff0a0e1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.682991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:01.726862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216861a7b0b7408cb6b4340ba5bab9b9', 'request_id': 'req-2a87d96e-014a-4290-901e-5decbc917b74'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:01.743094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.743905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.744216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.744487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.745929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] The token cache is being invalidate because project 216861a7b0b7408cb6b4340ba5bab9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:01.746370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.746624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:01.747980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2a87d96e-014a-4290-901e-5decbc917b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:01.748190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:01.751521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 932/1831] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:01 2022] DELETE /identity/v3/projects/216861a7b0b7408cb6b4340ba5bab9b9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:01.757587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe1b26-d3a3-450c-9332-b137dbd13fbf None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.758224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe1b26-d3a3-450c-9332-b137dbd13fbf None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.759189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe1b26-d3a3-450c-9332-b137dbd13fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.766514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 900/1832] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:01.768159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.770532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:01.790765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab14f9f-6e09-4466-b440-6f2af3f197b9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.798510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650024-c745-47d8-a375-275d68b2fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.799192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650024-c745-47d8-a375-275d68b2fc2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.799477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650024-c745-47d8-a375-275d68b2fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.799749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650024-c745-47d8-a375-275d68b2fc2f admin admin] PATH_INFO: `/v3/projects/a3b465f566d34d86b4c6acb60f965995/users/8cf5e398b68542599bf5f474dcd0f2de/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.802055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650024-c745-47d8-a375-275d68b2fc2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3b465f566d34d86b4c6acb60f965995, user_id=8cf5e398b68542599bf5f474dcd0f2de, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.805322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650024-c745-47d8-a375-275d68b2fc2f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:01.815630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:01.819846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-febe1b26-d3a3-450c-9332-b137dbd13fbf None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:01.822499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-febe1b26-d3a3-450c-9332-b137dbd13fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:01.836675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 901/1833] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:01 2022] PUT /identity/v3/projects/a3b465f566d34d86b4c6acb60f965995/users/8cf5e398b68542599bf5f474dcd0f2de/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:01.842958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.843282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.844099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.906636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] MFA Rules not processed for user `8cf5e398b68542599bf5f474dcd0f2de`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:01.908396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:01.921953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 933/1834] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:01.929552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.929847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.930282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.951873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:01.953179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:01.967966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 934/1835] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:01.972254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:01.982194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40531b54-9fa1-411a-acdb-f4ab12b03c4f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:01.986820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:01.987517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:01.987702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:01.987981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:01.988631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:01.988958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:01.990571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.000971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 902/1836] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:02.005048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 935/1837] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:02.018049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.020259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.029806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.039777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1e68e-7bc7-4472-a37a-a22a3ae805eb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.049036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.049703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.050242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.050691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.051348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.053478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.105281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74785cf-6a43-4ed9-8391-5dbc4576e031 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.109930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.110568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.110842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.111144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.120338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.121879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.131501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 936/1838] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:02.132292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.133599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.147392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1d128-0ac3-4f04-9cbc-6318a1634f94 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.152596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.154350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.154709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.154992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] PATH_INFO: `/v3/projects/02417a504b934caf9e1b2b8e7e903a1a` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.156224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02417a504b934caf9e1b2b8e7e903a1a)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.158160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.207854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 903/1839] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/projects => generated 389 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:02.209276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.212389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.288617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986926-4fe5-4b94-814f-faf9da027bc7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.305249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02417a504b934caf9e1b2b8e7e903a1a', 'request_id': 'req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:02.316398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02417a504b934caf9e1b2b8e7e903a1a', 'request_id': 'req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:02.367537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.368210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.368669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.368974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.373674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.403923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02417a504b934caf9e1b2b8e7e903a1a', 'request_id': 'req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:02.443238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] The token cache is being invalidate because project 02417a504b934caf9e1b2b8e7e903a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:02.443539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:02.448579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:02.451538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 937/1840] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:02 2022] DELETE /identity/v3/projects/02417a504b934caf9e1b2b8e7e903a1a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:02.455158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.456245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.459758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4832d8-1e21-4d7d-9013-4ed5c1524694 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.459994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.471638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 904/1841] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:02.485426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6fafc-5ad4-4b7e-8a3e-df9e10dd14a2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.485716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6fafc-5ad4-4b7e-8a3e-df9e10dd14a2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.486040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b6fafc-5ad4-4b7e-8a3e-df9e10dd14a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.507821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-46b6fafc-5ad4-4b7e-8a3e-df9e10dd14a2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:02.509203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-46b6fafc-5ad4-4b7e-8a3e-df9e10dd14a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.566148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe59b73-dab4-406c-a5f0-6eedfc2fac9f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.572245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.573386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.573672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.573948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.574589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.577116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.596134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 905/1842] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:02.598360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.598644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.599018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.623678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:02.625116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:02.634120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 906/1843] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:02.639055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.650192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396082c9-e1da-4018-8e9f-6788ec31a1ac None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.657590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.658265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.658559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.658813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.659510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:02.660381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.662123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.673159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 938/1844] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/users => generated 295 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:02.677352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.682500 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 907/1845] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:02.684842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.691932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cbc6d-71bc-4cf1-83aa-2e53a7d8e024 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.694958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd05ec-1e0d-4e76-a9cb-f72bca6f4e34 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.698546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.699317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.699605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.699984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.701002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.701611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.701828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.701949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.702156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.702794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.704011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.704955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.723023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 939/1846] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:02.727732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.739187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785cadb-4a20-4b96-ba03-efe6f72318af admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.744502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.745419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.745699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.745978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.747414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.749200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.761609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 940/1847] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:02.765703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.790261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b65af-b404-439a-b228-c6c8c21ce157 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.802821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79676610-0020-4207-9b2d-56f0d07b74f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.804109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79676610-0020-4207-9b2d-56f0d07b74f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.805899 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79676610-0020-4207-9b2d-56f0d07b74f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.817961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79676610-0020-4207-9b2d-56f0d07b74f0 admin admin] PATH_INFO: `/v3/projects/16073e4f658344e8965d00abe91c0876/users/890b717696494688b7b3df3b3f92077d/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.818082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 908/1848] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:02.822837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:02.852012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79676610-0020-4207-9b2d-56f0d07b74f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16073e4f658344e8965d00abe91c0876, user_id=890b717696494688b7b3df3b3f92077d, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.858976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79676610-0020-4207-9b2d-56f0d07b74f0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.911906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7045a-5659-4d06-8a0e-c39b5564c54f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:02.917661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:02.918300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.918779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.919707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:02.920595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:02.922876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:02.987129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 941/1849] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:02 2022] PUT /identity/v3/projects/16073e4f658344e8965d00abe91c0876/users/890b717696494688b7b3df3b3f92077d/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:02.998815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:02.999176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:02.999456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.049508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 909/1850] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:03.053775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.066913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] MFA Rules not processed for user `890b717696494688b7b3df3b3f92077d`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:03.070513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:03.089317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbdd0cd-0a52-4f76-9a3d-29fd82eee97b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.173720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 942/1851] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:03.179183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.196959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.197633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.197918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.198188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.199015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.201792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.220320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 910/1852] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:03.221411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.231567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23970b2-2647-418f-8cef-7672d0c14df4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.236675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.237319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.237597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.237867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.238667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.240212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.252588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 911/1853] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:03.256749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.267897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22850ec5-8658-47e7-abb3-2fcc2e271922 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.268097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd9739-8383-4819-8f06-a70a7bed4725 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.273606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20886c1c-2b8e-483b-a1ac-55d51fc640e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.273796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886c1c-2b8e-483b-a1ac-55d51fc640e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.274183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886c1c-2b8e-483b-a1ac-55d51fc640e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.274536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886c1c-2b8e-483b-a1ac-55d51fc640e5 admin admin] PATH_INFO: `/v3/projects/897289422ff14d5083e503ecfe39e848/users/35170640f56046debbbb5990a9a91f29/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.274862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.276110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.276540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.276988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.278478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.280642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.294300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20886c1c-2b8e-483b-a1ac-55d51fc640e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897289422ff14d5083e503ecfe39e848, user_id=35170640f56046debbbb5990a9a91f29, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.296374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20886c1c-2b8e-483b-a1ac-55d51fc640e5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.415221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 912/1854] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:03 2022] PUT /identity/v3/projects/897289422ff14d5083e503ecfe39e848/users/35170640f56046debbbb5990a9a91f29/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:03.426686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a48e-273d-49de-b321-870087f6da4a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.426972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a48e-273d-49de-b321-870087f6da4a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.427257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a48e-273d-49de-b321-870087f6da4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.430692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 943/1855] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:03 2022] POST /identity/v3/projects => generated 389 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:03.431803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.487244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c752a48e-273d-49de-b321-870087f6da4a None None] MFA Rules not processed for user `35170640f56046debbbb5990a9a91f29`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:03.495214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c752a48e-273d-49de-b321-870087f6da4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:03.547019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587a24d-5293-439b-8d25-e3144f4b9786 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.561163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.561759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.562042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.562409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.563056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.567316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.671069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 913/1856] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:03.675698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752a48e-273d-49de-b321-870087f6da4a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.686796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752a48e-273d-49de-b321-870087f6da4a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.691522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.692134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.692410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.692679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.693473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.695387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.719816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 944/1857] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:03 2022] POST /identity/v3/users => generated 301 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:03.724266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.734390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:03.737179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3792bdb-9800-4997-bbd0-908d250cc484 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.752310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.753100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.753473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.753823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.754578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.756352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.768477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 945/1858] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:03.772688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.783726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30404c10-1576-43ee-a402-eb6c0ad7dc3e admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.788437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.789346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.789439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.789685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] PATH_INFO: `/v3/projects/5ec8e56dacff42f6b87be22a6d959516/users/10543b59fc2f49e58631d1ac148ef847/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.808148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec8e56dacff42f6b87be22a6d959516, user_id=10543b59fc2f49e58631d1ac148ef847, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.810957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 914/1859] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:03 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:03.812117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.814823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.892835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03556baa-cd50-45cc-8652-3fb5a76c9984 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.897710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1948-9e61-4a2b-ab6a-04384566f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.898334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1948-9e61-4a2b-ab6a-04384566f2ff admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.898615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1948-9e61-4a2b-ab6a-04384566f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.898885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1948-9e61-4a2b-ab6a-04384566f2ff admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.899532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1948-9e61-4a2b-ab6a-04384566f2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:03.901553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1948-9e61-4a2b-ab6a-04384566f2ff admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:03.912664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 946/1860] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:03 2022] PUT /identity/v3/projects/5ec8e56dacff42f6b87be22a6d959516/users/10543b59fc2f49e58631d1ac148ef847/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:03.916462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:03.990130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786a297-4d01-4a5b-be17-e55a364a3f56 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:03.995617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:03.996173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:03.996456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:03.996757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:03.998011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.001672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.012822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 947/1861] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:04.016596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:04.028533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619dae7-2a99-439c-b859-9315c24cd608 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:04.033881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:04.034915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.035201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.035490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] PATH_INFO: `/v3/projects/5ec8e56dacff42f6b87be22a6d959516/users/10543b59fc2f49e58631d1ac148ef847/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.037866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ec8e56dacff42f6b87be22a6d959516, user_id=10543b59fc2f49e58631d1ac148ef847, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.040655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.055579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 948/1862] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:04 2022] PUT /identity/v3/projects/5ec8e56dacff42f6b87be22a6d959516/users/10543b59fc2f49e58631d1ac148ef847/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:04.065001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 915/1863] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:03 2022] POST /identity/v3/users => generated 299 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:04.070141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:04.072727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.073100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.073376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.105391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] MFA Rules not processed for user `10543b59fc2f49e58631d1ac148ef847`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:04.107409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:04.213829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d9437-3227-45dd-ad8e-ed798c5148af admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:04.220301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:04.221078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.221426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.221771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.222498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.224399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.241603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 949/1864] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:04.251525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:04.269177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4348bd0-2d96-4090-ad81-f133b0fc3d27 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:04.276008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:04.276361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.276649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.277260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] PATH_INFO: `/v3/projects/415fdd0431ab427e9ad12e8d7662e113/users/b9dbf6110c6f4742a6bb4d1588771b26/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.280849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 916/1865] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:04 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:04.300723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:04.302291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:04.321084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415fdd0431ab427e9ad12e8d7662e113, user_id=b9dbf6110c6f4742a6bb4d1588771b26, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.321084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.378880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874fe28-9d34-44a3-9ac7-10da27285b45 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:04.382943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:04.383669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.383874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.384156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.409946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.411418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.422309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 917/1866] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:04.435409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 950/1867] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:04 2022] PUT /identity/v3/projects/415fdd0431ab427e9ad12e8d7662e113/users/b9dbf6110c6f4742a6bb4d1588771b26/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:04.436776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:04.448311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991553e-c090-4251-8506-abd654fc3fe1 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:04.531538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0695765-0aad-4a43-b748-8b1230a97720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:04.531538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0695765-0aad-4a43-b748-8b1230a97720 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.531538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0695765-0aad-4a43-b748-8b1230a97720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.531538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0695765-0aad-4a43-b748-8b1230a97720 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.532384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0695765-0aad-4a43-b748-8b1230a97720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.532384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0695765-0aad-4a43-b748-8b1230a97720 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.547592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 918/1868] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:04.560258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:04.567390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cd64c-1d07-44f1-b48b-4237d514ea33 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:04.575908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:04.576895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.577392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.577897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] PATH_INFO: `/v3/projects/415fdd0431ab427e9ad12e8d7662e113/users/b9dbf6110c6f4742a6bb4d1588771b26/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.580224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415fdd0431ab427e9ad12e8d7662e113, user_id=b9dbf6110c6f4742a6bb4d1588771b26, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:04.584200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:04.619053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:04.619640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:04.620083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:04.623857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 951/1869] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:04 2022] PUT /identity/v3/projects/415fdd0431ab427e9ad12e8d7662e113/users/b9dbf6110c6f4742a6bb4d1588771b26/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:04.659285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] MFA Rules not processed for user `b9dbf6110c6f4742a6bb4d1588771b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:04.660528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:04.769323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 919/1870] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:04.774493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:04.974096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:05.047080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c781-b1b1-4c8a-92d6-86d584df03f8 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:05.051650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f09e4f-4f2b-40bd-a8d0-32d3e767efbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:05.052262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f09e4f-4f2b-40bd-a8d0-32d3e767efbd service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:05.052543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f09e4f-4f2b-40bd-a8d0-32d3e767efbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:05.053408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f09e4f-4f2b-40bd-a8d0-32d3e767efbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:05.063791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f09e4f-4f2b-40bd-a8d0-32d3e767efbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:05.065279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f09e4f-4f2b-40bd-a8d0-32d3e767efbd service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:05.086076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 952/1871] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:05.423882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:05.530411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd106a-16ef-4fc3-925d-97284dc4e3a3 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:05.535805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:05.536524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:05.536895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:05.537248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] PATH_INFO: `/v3/users/890b717696494688b7b3df3b3f92077d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:05.538581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=890b717696494688b7b3df3b3f92077d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:05.540849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:05.877515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890b717696494688b7b3df3b3f92077d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.878343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890b717696494688b7b3df3b3f92077d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.890503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890b717696494688b7b3df3b3f92077d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.897468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890b717696494688b7b3df3b3f92077d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.030607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890b717696494688b7b3df3b3f92077d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.052449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 920/1872] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:05 2022] DELETE /identity/v3/users/890b717696494688b7b3df3b3f92077d => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:06.055525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.056496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.057063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.093382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:06.094846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.188372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 953/1873] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:06.198115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.199634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.256612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5189721-783c-40ee-9095-fedd6bbcd403 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.260248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.260851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.261157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.261428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.271557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.273107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.300034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 921/1874] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:06.759651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.761984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.797157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363c849-3793-46ac-92b0-6cf1d898c94a None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.803046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.803809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.804227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.804508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] PATH_INFO: `/v3/users/35170640f56046debbbb5990a9a91f29` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.805808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35170640f56046debbbb5990a9a91f29)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.807891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:07.034478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35170640f56046debbbb5990a9a91f29'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.035288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35170640f56046debbbb5990a9a91f29'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.043355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35170640f56046debbbb5990a9a91f29'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.045034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35170640f56046debbbb5990a9a91f29'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.173694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5b07ed3b-e27c-4d15-a566-80d7f86e2273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35170640f56046debbbb5990a9a91f29'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.175684 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:07.187481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:07.190871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:07.218921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110e7f1-d66b-4dc0-b363-48f9a17a4cd3 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:07.223395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 954/1875] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:06 2022] DELETE /identity/v3/users/35170640f56046debbbb5990a9a91f29 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:07.227490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:07.228345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:07.229230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:07.267479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:07.269553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:07.320397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:07.321260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:07.322131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:07.322131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] PATH_INFO: `/v3/projects/16073e4f658344e8965d00abe91c0876` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:07.323255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=16073e4f658344e8965d00abe91c0876)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:07.325759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:07.359863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 955/1876] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:07.369467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:07.371012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:07.434478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2edf7ef-5a9b-4e51-9d75-b1b5741e2a2c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:07.441618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:07.442600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:07.442903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:07.443292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:07.460418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:07.462063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:07.473974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16073e4f658344e8965d00abe91c0876', 'request_id': 'req-d56e6b31-b9bb-48e6-933f-c126822df65e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.477583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 956/1877] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:07.483781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16073e4f658344e8965d00abe91c0876', 'request_id': 'req-d56e6b31-b9bb-48e6-933f-c126822df65e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.494871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16073e4f658344e8965d00abe91c0876', 'request_id': 'req-d56e6b31-b9bb-48e6-933f-c126822df65e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.581422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] The token cache is being invalidate because project 16073e4f658344e8965d00abe91c0876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:07.581730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.582597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.584639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 922/1878] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:07 2022] DELETE /identity/v3/projects/16073e4f658344e8965d00abe91c0876 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:07.588730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:07.590247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:07.603533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b77ffe-0fd3-46a7-ad28-b0670f8966ad service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:07.669865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:07.670530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:07.670812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:07.671090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] PATH_INFO: `/v3/users/10543b59fc2f49e58631d1ac148ef847` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:07.672287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] RBAC: Authorizing `identity:delete_user(user_id=10543b59fc2f49e58631d1ac148ef847)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:07.674303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:07.935577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:07.938374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10543b59fc2f49e58631d1ac148ef847'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.939278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10543b59fc2f49e58631d1ac148ef847'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.948537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10543b59fc2f49e58631d1ac148ef847'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:07.949383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10543b59fc2f49e58631d1ac148ef847'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.111380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10543b59fc2f49e58631d1ac148ef847'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.151767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 957/1879] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:07 2022] DELETE /identity/v3/users/10543b59fc2f49e58631d1ac148ef847 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:08.279866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:08.281814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:08.316889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e6b31-b9bb-48e6-933f-c126822df65e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:08.446572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:08.448670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:08.454538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:08.454538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] PATH_INFO: `/v3/projects/897289422ff14d5083e503ecfe39e848` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:08.454538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897289422ff14d5083e503ecfe39e848)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:08.455293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:08.695127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897289422ff14d5083e503ecfe39e848', 'request_id': 'req-e6d9d0ba-4416-4d94-bda3-e26cad357e16'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.703018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897289422ff14d5083e503ecfe39e848', 'request_id': 'req-e6d9d0ba-4416-4d94-bda3-e26cad357e16'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.710589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897289422ff14d5083e503ecfe39e848', 'request_id': 'req-e6d9d0ba-4416-4d94-bda3-e26cad357e16'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.852597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:08.854768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:08.868446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40ddb5-38ed-43a8-b2b6-1df536fd784f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:08.921439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] The token cache is being invalidate because project 897289422ff14d5083e503ecfe39e848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:08.922662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.928883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:08.946611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 923/1880] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:08 2022] DELETE /identity/v3/projects/897289422ff14d5083e503ecfe39e848 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:08.947596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:08.952538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:08.988784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:08.989587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:08.990428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:08.990428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] PATH_INFO: `/v3/users/8d7c0f34fbdc404593f84f48d9e4e868` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:08.992294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7c0f34fbdc404593f84f48d9e4e868)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:08.996462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:09.065458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9d0ba-4416-4d94-bda3-e26cad357e16 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:09.070143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:09.070838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:09.071117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:09.071389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] PATH_INFO: `/v3/users/b9dbf6110c6f4742a6bb4d1588771b26` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:09.072678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9dbf6110c6f4742a6bb4d1588771b26)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:09.074716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:09.284851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7c0f34fbdc404593f84f48d9e4e868'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.285639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7c0f34fbdc404593f84f48d9e4e868'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.292499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7c0f34fbdc404593f84f48d9e4e868'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.293182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7c0f34fbdc404593f84f48d9e4e868'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.312282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9dbf6110c6f4742a6bb4d1588771b26'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.313093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9dbf6110c6f4742a6bb4d1588771b26'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.320578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9dbf6110c6f4742a6bb4d1588771b26'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.321246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9dbf6110c6f4742a6bb4d1588771b26'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.419548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7c0f34fbdc404593f84f48d9e4e868'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.440055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 958/1881] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:08 2022] DELETE /identity/v3/users/8d7c0f34fbdc404593f84f48d9e4e868 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:09.441016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:09.442436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:09.516632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5fceb-29db-4076-ae4a-a13684d78cbb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:09.535434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:09.535434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:09.535434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:09.535434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] PATH_INFO: `/v3/projects/5ec8e56dacff42f6b87be22a6d959516` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:09.535967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ec8e56dacff42f6b87be22a6d959516)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:09.535967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:09.562101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4bdb9af0-16cd-477c-9dbe-51cfa1c6562a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9dbf6110c6f4742a6bb4d1588771b26'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.581520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 924/1882] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:08 2022] DELETE /identity/v3/users/b9dbf6110c6f4742a6bb4d1588771b26 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:09.584853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:09.584853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:09.584853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:09.607226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:09.614221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:09.718540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 925/1883] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:09.751883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ec8e56dacff42f6b87be22a6d959516', 'request_id': 'req-21c935b0-c21b-4a59-b4db-d992264c2662'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.758293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:09.759500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ec8e56dacff42f6b87be22a6d959516', 'request_id': 'req-21c935b0-c21b-4a59-b4db-d992264c2662'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.764919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:09.837100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d36ff-d4a6-467c-8535-4b92be552482 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:09.876223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ec8e56dacff42f6b87be22a6d959516', 'request_id': 'req-21c935b0-c21b-4a59-b4db-d992264c2662'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.898322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] The token cache is being invalidate because project 5ec8e56dacff42f6b87be22a6d959516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:09.898322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.898322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-21c935b0-c21b-4a59-b4db-d992264c2662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:09.901295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 959/1884] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:09 2022] DELETE /identity/v3/projects/5ec8e56dacff42f6b87be22a6d959516 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:09.907448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:09.908026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:09.908471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:09.908746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:09.909663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:09.910001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:09.910345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:09.912243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:09.935382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:09.937415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:10.002597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.004124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24f9b5-bcad-4883-aff8-a74573494b6c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.019015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 926/1885] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:10.023852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 960/1886] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:10.034128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.034128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.034128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.074238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:10.077431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:10.093229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 927/1887] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:10.097981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.109946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498bf1e-5357-40d2-8d2d-6d2849d2ea7f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.116194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.116825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.117142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.117423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.118084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:10.118548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.120186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.133417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 961/1888] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:10.141938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.152019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b787830-5568-4d2e-a284-ffd74394013c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.157313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.157983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.158269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.158546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.159356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.161483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.298809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 928/1889] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:10 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:10.300359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.326158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679cedb6-7f11-48b3-8cee-e8c5337a1ed2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.385670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.386287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.386571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.386847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.388081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.390284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.456642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.460247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:10.486354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de972da-150a-4667-a905-0db540bcb852 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.492244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 962/1890] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:10 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:10.497647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.508466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:10.521923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.521923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.522429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.524471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] PATH_INFO: `/v3/projects/415fdd0431ab427e9ad12e8d7662e113` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.527488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=415fdd0431ab427e9ad12e8d7662e113)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.529920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.532416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c760d-1fd3-4eac-9c35-15e03b7f1aed admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.541496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.543101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.544201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.544951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.545976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.547801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.560945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 963/1891] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:10.565670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.577093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951d2a6-a902-4b57-abdc-a20a7a4efaa0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.582753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.586057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.588554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.588554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.588554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.589723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.601444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 964/1892] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:10.605275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.618415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17371189-33d5-457d-9b45-3e770a45b347 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:10.624233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c3810-f236-486f-ab9a-a67e8b1ed012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:10.625021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c3810-f236-486f-ab9a-a67e8b1ed012 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.625376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c3810-f236-486f-ab9a-a67e8b1ed012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.625740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c3810-f236-486f-ab9a-a67e8b1ed012 admin admin] PATH_INFO: `/v3/projects/e0eb8bfcee2644b59b890c87e2bee176/users/50ebe27c3dba443983c4dd248ea10f41/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.647433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c3810-f236-486f-ab9a-a67e8b1ed012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0eb8bfcee2644b59b890c87e2bee176, user_id=50ebe27c3dba443983c4dd248ea10f41, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:10.650425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c3810-f236-486f-ab9a-a67e8b1ed012 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:10.802878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 965/1893] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:10 2022] PUT /identity/v3/projects/e0eb8bfcee2644b59b890c87e2bee176/users/50ebe27c3dba443983c4dd248ea10f41/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:10.816028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:10.816563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:10.816848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:10.818358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415fdd0431ab427e9ad12e8d7662e113', 'request_id': 'req-d1b068dd-edea-44ac-b311-8ff387982b25'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:10.825553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415fdd0431ab427e9ad12e8d7662e113', 'request_id': 'req-d1b068dd-edea-44ac-b311-8ff387982b25'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:10.835800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415fdd0431ab427e9ad12e8d7662e113', 'request_id': 'req-d1b068dd-edea-44ac-b311-8ff387982b25'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:10.870914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] MFA Rules not processed for user `50ebe27c3dba443983c4dd248ea10f41`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:10.870914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:10.939944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 966/1894] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:10.940800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.942097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:10.962172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] The token cache is being invalidate because project 415fdd0431ab427e9ad12e8d7662e113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:10.962172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:10.963324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:10.967399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 929/1895] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:10 2022] DELETE /identity/v3/projects/415fdd0431ab427e9ad12e8d7662e113 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:10.967399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:10.970989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.016830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaffeb-5682-4de6-ae27-0fd1e0b69f9b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.022999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.023648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.023927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.024629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] PATH_INFO: `/v3/projects/1a62bf6be8d440edbf3e70d7cc897c41` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.026038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a62bf6be8d440edbf3e70d7cc897c41)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.030117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.052812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b068dd-edea-44ac-b311-8ff387982b25 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.059719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442f42d-84c6-4f53-a0d0-11ea351b5d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.059719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442f42d-84c6-4f53-a0d0-11ea351b5d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.059719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442f42d-84c6-4f53-a0d0-11ea351b5d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.060085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442f42d-84c6-4f53-a0d0-11ea351b5d50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.060775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442f42d-84c6-4f53-a0d0-11ea351b5d50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.062910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442f42d-84c6-4f53-a0d0-11ea351b5d50 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.175400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a62bf6be8d440edbf3e70d7cc897c41', 'request_id': 'req-2baadc42-af69-4785-8980-3b64d7a24525'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.184664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a62bf6be8d440edbf3e70d7cc897c41', 'request_id': 'req-2baadc42-af69-4785-8980-3b64d7a24525'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.202773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 930/1896] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:10 2022] POST /identity/v3/projects => generated 367 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:11.204151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.204439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.204713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.225448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:11.226669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.243790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:11.270688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a62bf6be8d440edbf3e70d7cc897c41', 'request_id': 'req-2baadc42-af69-4785-8980-3b64d7a24525'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.282192 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] The token cache is being invalidate because project 1a62bf6be8d440edbf3e70d7cc897c41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:11.282917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.288294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.290336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 967/1897] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:10 2022] DELETE /identity/v3/projects/1a62bf6be8d440edbf3e70d7cc897c41 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:11.291108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.292630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.312230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 931/1898] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:11.313808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.317870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.335193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a66237-725c-4ef3-a582-579665df6d08 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.343605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.344336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.344682 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.345392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] PATH_INFO: `/v3/users/8cf5e398b68542599bf5f474dcd0f2de` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.347331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf5e398b68542599bf5f474dcd0f2de)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.349571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.370680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baadc42-af69-4785-8980-3b64d7a24525 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.376467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a37b8-4726-4ada-8f9c-1c39b1fb5f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.377353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a37b8-4726-4ada-8f9c-1c39b1fb5f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.377681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a37b8-4726-4ada-8f9c-1c39b1fb5f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.378082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a37b8-4726-4ada-8f9c-1c39b1fb5f1d admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.378936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a37b8-4726-4ada-8f9c-1c39b1fb5f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.381005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a37b8-4726-4ada-8f9c-1c39b1fb5f1d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.500519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 968/1899] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:11 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:11.504851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.505169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.505460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.526478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:11.528649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.536543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 969/1900] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:11.537615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.548376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93590d-1fbc-4067-a3be-b4886084b697 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.553433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.554079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.554363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.554638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.555485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.556919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.569052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 970/1901] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:11.570052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.581438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190767f4-9a64-45d4-b6a9-ab395721fbc0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.590318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.594029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.594029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.594029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.594727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:11.595280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.597446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.603740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf5e398b68542599bf5f474dcd0f2de'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.604457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf5e398b68542599bf5f474dcd0f2de'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.619215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf5e398b68542599bf5f474dcd0f2de'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.620059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf5e398b68542599bf5f474dcd0f2de'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.629029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 971/1902] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:11.629691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.637553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.651123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3bff-be7d-4fe4-92c4-ed7e0b6b1da8 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.659678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.660367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.669948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.669948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] PATH_INFO: `/v3/projects/f9fc36caf1b340aeb08d45d6fcfca194/users/7c1159270cd0426bab54ad4fbe9e2914/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.680334 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9fc36caf1b340aeb08d45d6fcfca194, user_id=7c1159270cd0426bab54ad4fbe9e2914, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.683543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.756540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf5e398b68542599bf5f474dcd0f2de'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:11.780183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 932/1903] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:11 2022] DELETE /identity/v3/users/8cf5e398b68542599bf5f474dcd0f2de => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:11.781104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.782523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:11.791487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 972/1904] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:11 2022] PUT /identity/v3/projects/f9fc36caf1b340aeb08d45d6fcfca194/users/7c1159270cd0426bab54ad4fbe9e2914/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:11.796168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.887752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb7309-1e43-44ba-90d7-6e56b12fe63b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.893027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.893912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.894403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.894529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9c954-74c5-40c9-8b72-c52264536d7b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.895106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.895973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.897918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.900301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.901156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.901599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.902017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.904012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.906540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.917905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 973/1905] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:11.922273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:11.932833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4509ab3-ab2e-4d2d-be79-b5941c37fbae admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:11.940135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892db48d-8e41-4e39-8f53-697b9e12f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:11.940920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db48d-8e41-4e39-8f53-697b9e12f5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.941208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db48d-8e41-4e39-8f53-697b9e12f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.941605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892db48d-8e41-4e39-8f53-697b9e12f5dd admin admin] PATH_INFO: `/v3/projects/f9fc36caf1b340aeb08d45d6fcfca194/users/7c1159270cd0426bab54ad4fbe9e2914/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:11.943847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892db48d-8e41-4e39-8f53-697b9e12f5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9fc36caf1b340aeb08d45d6fcfca194, user_id=7c1159270cd0426bab54ad4fbe9e2914, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:11.946616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892db48d-8e41-4e39-8f53-697b9e12f5dd admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:11.957561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 974/1906] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:11 2022] PUT /identity/v3/projects/f9fc36caf1b340aeb08d45d6fcfca194/users/7c1159270cd0426bab54ad4fbe9e2914/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:11.969218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:11.969508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:11.969789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.001162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] MFA Rules not processed for user `7c1159270cd0426bab54ad4fbe9e2914`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:12.002431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.054477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 933/1907] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:11 2022] POST /identity/v3/projects => generated 373 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:12.055401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.134532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=79,keystone:INSERT=4,keystone:DELETE=17 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:12.150158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 975/1908] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:12.150875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c6811-9ee6-46ab-9be5-84d5787593f8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.160207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.160207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.160207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.160207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.161052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.162997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.165082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.167413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.260533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2c98-b036-4bd0-95b8-e35a32faf2f0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.276902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 934/1909] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:12 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:12.283128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.294294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc680a6-e26f-4a79-9612-f7cfdd476473 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.299384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.300152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.300445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.300736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.301404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.303018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.318704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.319532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.319993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.320487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.337529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 935/1910] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:12.343255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.354179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f56328-d985-4b85-bb66-edec041809e2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.359537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.360323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.360628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.360925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.366327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.366327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.377190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 936/1911] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:12.378204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.389376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24a884-bd46-4b1d-8b52-bfc32a1032a7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.395022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.395159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013dac7f-1f00-4eb4-8abe-53034ca5ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.395686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dac7f-1f00-4eb4-8abe-53034ca5ea6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.395974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dac7f-1f00-4eb4-8abe-53034ca5ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.396118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.396365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013dac7f-1f00-4eb4-8abe-53034ca5ea6d admin admin] PATH_INFO: `/v3/projects/4fadeccf48f8447e8d56ebdffdaecea6/users/bf909a51728a44948fd5f86faaf58c24/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.411538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 976/1912] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:12.414674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.416241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.416376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dac7f-1f00-4eb4-8abe-53034ca5ea6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fadeccf48f8447e8d56ebdffdaecea6, user_id=bf909a51728a44948fd5f86faaf58c24, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.423647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013dac7f-1f00-4eb4-8abe-53034ca5ea6d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.430811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c022-0ba6-40c7-ac81-80439e04e74e service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.438537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.439116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.439594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.440083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] PATH_INFO: `/v3/projects/a3b465f566d34d86b4c6acb60f965995` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.444077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3b465f566d34d86b4c6acb60f965995)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.447809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.522685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 937/1913] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:12 2022] PUT /identity/v3/projects/4fadeccf48f8447e8d56ebdffdaecea6/users/bf909a51728a44948fd5f86faaf58c24/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:12.533667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.533952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.534222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.568024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] MFA Rules not processed for user `bf909a51728a44948fd5f86faaf58c24`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:12.569334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.625912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3b465f566d34d86b4c6acb60f965995', 'request_id': 'req-f656f781-d870-4a43-b89e-f0128f5acf5b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:12.633116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3b465f566d34d86b4c6acb60f965995', 'request_id': 'req-f656f781-d870-4a43-b89e-f0128f5acf5b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:12.644065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 938/1914] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:12.655526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.710195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be48209-d2aa-4ebf-85f3-8b8d82da8b87 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.745100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3b465f566d34d86b4c6acb60f965995', 'request_id': 'req-f656f781-d870-4a43-b89e-f0128f5acf5b'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:12.757179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] The token cache is being invalidate because project a3b465f566d34d86b4c6acb60f965995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:12.757473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:12.758254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:12.759724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 977/1915] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:12 2022] DELETE /identity/v3/projects/a3b465f566d34d86b4c6acb60f965995 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:12.760492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.761360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.761805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.761997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.763267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.763544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.765498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.774353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f781-d870-4a43-b89e-f0128f5acf5b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.778144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.778788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.779092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.779370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.781087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.825701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.827064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a7ca5-4eae-4e80-b4cd-a5d41d997918 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.843189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.844685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 939/1916] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:12.844821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.846741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfec90a-9927-445f-b937-9d54c6aa95ac None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.847367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfec90a-9927-445f-b937-9d54c6aa95ac None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.847655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfec90a-9927-445f-b937-9d54c6aa95ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.855327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 978/1917] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:12.873068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:12.875369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.889432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5bfec90a-9927-445f-b937-9d54c6aa95ac None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:12.891241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5bfec90a-9927-445f-b937-9d54c6aa95ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:12.948951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5578077-0be0-49e0-bf8f-0908d80750a5 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:12.954266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:12.954902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.955174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.955449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] PATH_INFO: `/v3/users/bf909a51728a44948fd5f86faaf58c24` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.956679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf909a51728a44948fd5f86faaf58c24)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:12.958972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:12.960728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 940/1918] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:12.969523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:12.970063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:12.970530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:12.999115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:13.000343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:13.008757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 941/1919] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:13.013145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.022223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710beb1-61a7-4ce5-8c02-0e381ea9b52a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.026883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.027465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.027798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.028066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.028739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:13.029067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.030642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.041829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 942/1920] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:13.046650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.055917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd488e7-6050-4680-b30e-cb3f583f038d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.061577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.062148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.062445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.062746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.063422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.068477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.164977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf909a51728a44948fd5f86faaf58c24'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:13.165723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf909a51728a44948fd5f86faaf58c24'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:13.173444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf909a51728a44948fd5f86faaf58c24'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:13.174071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf909a51728a44948fd5f86faaf58c24'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:13.190924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 943/1921] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:13 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:13.192274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.194707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:13.264805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df21ff-178d-40b5-a9ff-3880d518ca6b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.269666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.274953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.274953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.274953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.274953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.274953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.282188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7f87bac5-dbf7-4d0d-9346-a6a8b2329e1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf909a51728a44948fd5f86faaf58c24'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:13.307734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 979/1922] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:12 2022] DELETE /identity/v3/users/bf909a51728a44948fd5f86faaf58c24 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:13.312063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.312350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.312626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.332066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:13.333583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:13.356987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:13.394839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 980/1923] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:13.407794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.409121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:13.428285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 944/1924] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:13 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:13.429670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.443121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608cceb-10e8-4b23-b1d5-4e5c432219db admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.449274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.449974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.450279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.450548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.451191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.452840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.463200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 945/1925] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:13.467633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.473182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce5f4d-4dc7-45b5-9252-5e16910dc63f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.477106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6949f-e585-4db6-b058-e2b16c84aa34 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.477425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545f618-5701-495e-ad4d-95deb96fc8ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.478040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545f618-5701-495e-ad4d-95deb96fc8ab service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.478319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545f618-5701-495e-ad4d-95deb96fc8ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.478596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545f618-5701-495e-ad4d-95deb96fc8ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.482404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.482986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.483264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.483562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.484209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.485813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.490652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545f618-5701-495e-ad4d-95deb96fc8ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.490652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545f618-5701-495e-ad4d-95deb96fc8ab service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.501921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 946/1926] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:13.502027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 981/1927] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:13.502398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.512399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9194b30d-2478-4a95-8a8f-8d6a082dcfd3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.517404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.518060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.518373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.518598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] PATH_INFO: `/v3/projects/40aedd1efdbb473bbfa27fe28dc25df1/users/ac677a0592a94d90b469b0a533e12fd1/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.537082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40aedd1efdbb473bbfa27fe28dc25df1, user_id=ac677a0592a94d90b469b0a533e12fd1, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.538669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.660805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 947/1928] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:13 2022] PUT /identity/v3/projects/40aedd1efdbb473bbfa27fe28dc25df1/users/ac677a0592a94d90b469b0a533e12fd1/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:13.675485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.675775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.676054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.716808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] MFA Rules not processed for user `ac677a0592a94d90b469b0a533e12fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:13.718730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:13.819143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 982/1929] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:13.823837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.889468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5ca18-3546-4168-b637-80b524e3f360 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:13.894114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:13.894722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:13.895001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:13.895274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:13.895916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:13.897979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:13.945121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:13.991709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:13.993781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 948/1930] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:13 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:13.994532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:14.001270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.014813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addce7e-0d1b-4c3b-ac83-012596b1a914 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.072307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1b87a-b02a-41d7-83d5-126df17bedf6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.077974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.078587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.078869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.079176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.079820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.081960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.082720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.083621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.083926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.084201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] PATH_INFO: `/v3/projects/4fadeccf48f8447e8d56ebdffdaecea6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.085413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fadeccf48f8447e8d56ebdffdaecea6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.087331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.202562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 949/1931] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:14.206324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.216101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aec01-37ed-4be8-b5d1-ba79bcdb4cfa admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.222041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.222632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.222916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.223194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.223838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.225928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.238959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 950/1932] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:14.243709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.247257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fadeccf48f8447e8d56ebdffdaecea6', 'request_id': 'req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:14.254686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fadeccf48f8447e8d56ebdffdaecea6', 'request_id': 'req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:14.259484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb8ee3-975b-4f12-98dc-d63572508aa8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.320016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.320623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.320927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.321202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.321871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.323472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.336966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 951/1933] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:14.337931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.347740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9f3e-8e7e-4bda-b43c-e87fb78b5acf admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.350485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fadeccf48f8447e8d56ebdffdaecea6', 'request_id': 'req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:14.352593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d66092-48bc-4a2c-b756-7ce68b1ec263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.353269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d66092-48bc-4a2c-b756-7ce68b1ec263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.353552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d66092-48bc-4a2c-b756-7ce68b1ec263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.353808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d66092-48bc-4a2c-b756-7ce68b1ec263 admin admin] PATH_INFO: `/v3/projects/1f91058ec6384c9681d02715643517e1/users/3ac6c9bd3bf2425c9d0d058d6adf7de1/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.367146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] The token cache is being invalidate because project 4fadeccf48f8447e8d56ebdffdaecea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:14.367409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:14.368343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d6adfbb0-b28b-45a3-b642-1d15aa775cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:14.370094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 983/1934] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:13 2022] DELETE /identity/v3/projects/4fadeccf48f8447e8d56ebdffdaecea6 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:14.372163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d66092-48bc-4a2c-b756-7ce68b1ec263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f91058ec6384c9681d02715643517e1, user_id=3ac6c9bd3bf2425c9d0d058d6adf7de1, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.375013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d66092-48bc-4a2c-b756-7ce68b1ec263 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.375719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c686a-e361-4487-abb6-32bb6a9d5b01 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.376008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c686a-e361-4487-abb6-32bb6a9d5b01 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.376338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c686a-e361-4487-abb6-32bb6a9d5b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.397146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ba4c686a-e361-4487-abb6-32bb6a9d5b01 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:14.398371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ba4c686a-e361-4487-abb6-32bb6a9d5b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:14.406737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 984/1935] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:14.414849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.415152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.417723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.435915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:14.437127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:14.446177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 985/1936] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:14.452185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 952/1937] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:14 2022] PUT /identity/v3/projects/1f91058ec6384c9681d02715643517e1/users/3ac6c9bd3bf2425c9d0d058d6adf7de1/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:14.452322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.459312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.459533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a9f9-a802-4fd6-8c60-de33addecc68 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.459633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.459811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.488691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] MFA Rules not processed for user `3ac6c9bd3bf2425c9d0d058d6adf7de1`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:14.490017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:14.516548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.517187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.517802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.517802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.518370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:14.518675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.521194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.534030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 986/1938] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:14.538495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.549624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20a49c-35a5-467f-8eb8-53d0a9bd81fc admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.554418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.555055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.555329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.558069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.558069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.558387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 953/1939] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:14.560729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.563609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.565046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:14.578667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ba7c8-427a-4df8-8d84-0598ea85efdb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.583634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.584251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.584533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.584808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.585470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.587397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.700780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 987/1940] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:14.704949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.732799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 954/1941] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:14.738359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.773060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29950c-7922-4e11-942e-47d919a4956a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.809968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfb7da-1afb-4ea8-ad1d-64eaa29f64a6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.816377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.819301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.819772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.820149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.820924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.823267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.850260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.850889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.851173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.851452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.852151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.854176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.860766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:14.956429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 955/1942] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:14.957723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:14.969706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06437199-4c16-441c-b4f6-986128cece33 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:14.974492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:14.975152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:14.975431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:14.975759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:14.976460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:14.978124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:14.997651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 956/1943] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:14.999489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.004662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 988/1944] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:14 2022] POST /identity/v3/users => generated 288 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:15.005760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.014327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a1066a-6137-41f8-8196-b2be59aca1b2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.016390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b66c8-c2d8-4255-bbaa-61801d437f77 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.019862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.020656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.020977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.021258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] PATH_INFO: `/v3/projects/0567c02ab6514212a9a3e7b25742a40c/users/c0fb3c1b50e84de78d2541e1fe91a712/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.022206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.022776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.023061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.023339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.023988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.025654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.039199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0567c02ab6514212a9a3e7b25742a40c, user_id=c0fb3c1b50e84de78d2541e1fe91a712, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.045028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 989/1945] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:15.045282 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.046570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.057104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6188d6-19ef-4f07-846c-1f31be30b55a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.061947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.062572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.062854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.063130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.063814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.065481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.077382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 990/1946] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:15.084088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.095128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5393-972e-4bdb-8b43-b513fb6ce277 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.100153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5b9e2-c9c5-49a6-952c-90c483c3a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.100812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5b9e2-c9c5-49a6-952c-90c483c3a8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.101120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5b9e2-c9c5-49a6-952c-90c483c3a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.101399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5b9e2-c9c5-49a6-952c-90c483c3a8a4 admin admin] PATH_INFO: `/v3/projects/486e089e13754a0eb90b15f601bc9d8f/users/4fa5865077184c1eb5468c44ec60e650/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.116589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5b9e2-c9c5-49a6-952c-90c483c3a8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486e089e13754a0eb90b15f601bc9d8f, user_id=4fa5865077184c1eb5468c44ec60e650, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.119307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5b9e2-c9c5-49a6-952c-90c483c3a8a4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.147558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 957/1947] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:14 2022] PUT /identity/v3/projects/0567c02ab6514212a9a3e7b25742a40c/users/c0fb3c1b50e84de78d2541e1fe91a712/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:15.151720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.213829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 991/1948] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:15 2022] PUT /identity/v3/projects/486e089e13754a0eb90b15f601bc9d8f/users/4fa5865077184c1eb5468c44ec60e650/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:15.215131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bf7dd-8419-41c8-9ee3-83bb30d08091 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.224966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea96f0f-3ee5-4bc7-8542-0bd047728a7b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.225236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea96f0f-3ee5-4bc7-8542-0bd047728a7b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.225531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea96f0f-3ee5-4bc7-8542-0bd047728a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.253567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cea96f0f-3ee5-4bc7-8542-0bd047728a7b None None] MFA Rules not processed for user `4fa5865077184c1eb5468c44ec60e650`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:15.257970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cea96f0f-3ee5-4bc7-8542-0bd047728a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:15.274792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.275446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.275726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.275996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.276733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.279040 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.289155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 958/1949] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:15.293478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.304360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162bb6c-4bac-4e19-b7b8-f15fa1a0d3ab admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.309349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.310093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.310383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.310748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] PATH_INFO: `/v3/projects/0567c02ab6514212a9a3e7b25742a40c/users/c0fb3c1b50e84de78d2541e1fe91a712/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.313536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0567c02ab6514212a9a3e7b25742a40c, user_id=c0fb3c1b50e84de78d2541e1fe91a712, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.316432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.326301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 992/1950] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:15.328351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 959/1951] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:15 2022] PUT /identity/v3/projects/0567c02ab6514212a9a3e7b25742a40c/users/c0fb3c1b50e84de78d2541e1fe91a712/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:15.338619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.338913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.339193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.344542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.346445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:15.374728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] MFA Rules not processed for user `c0fb3c1b50e84de78d2541e1fe91a712`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:15.376225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:15.409014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada116a2-daa2-43b6-826e-20fc3dbd11b8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.413117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.414244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.414244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.414544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.476976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 993/1952] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:15.485865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.488202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.494060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:15.500458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 960/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:15.511905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.514136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:15.526703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad6651a-9730-47b3-bef6-5b09e39591c8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.531026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.531689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.531978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.532251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.541793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.543407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.553825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 994/1954] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:15.691779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:15.701984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f6d06-bad1-4d65-870e-ea45d1cf3aec service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:15.705941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:15.706566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:15.706842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:15.707110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:15.709191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:15.771058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:15.772531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64052c2b-0956-4ad8-89f0-248d15a53893 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:15.782781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 961/1955] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:17.053717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:17.133689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:18.882445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:18.939129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:20.470977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:20.478352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:20.544358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531a89a-2b92-4964-b26d-5dff3f204bc1 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:20.549507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:20.550150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:20.550426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:20.550702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] PATH_INFO: `/v3/users/50ebe27c3dba443983c4dd248ea10f41` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:20.556252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ebe27c3dba443983c4dd248ea10f41)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:20.559166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:20.630647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:20.792453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ebe27c3dba443983c4dd248ea10f41'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:20.793141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ebe27c3dba443983c4dd248ea10f41'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:20.799897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ebe27c3dba443983c4dd248ea10f41'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:20.800552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ebe27c3dba443983c4dd248ea10f41'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:20.881682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ebe27c3dba443983c4dd248ea10f41'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:20.900918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 995/1956] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:20 2022] DELETE /identity/v3/users/50ebe27c3dba443983c4dd248ea10f41 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:20.906198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:20.906489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:20.906781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:20.932035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:20.933018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:20.997796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 962/1957] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:21.009297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:21.011040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:21.086248 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3741c08-739d-4750-9c0c-8e0d74e246ef admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:21.090814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:21.091487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:21.091782 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:21.091994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:21.101762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:21.103304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:21.113555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 996/1958] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:21.282306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:21.532696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:21.534176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:21.546662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5abca-06d4-4878-b329-b009c4090aaa None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:21.551540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:21.552162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:21.552437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:21.552707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] PATH_INFO: `/v3/users/4fa5865077184c1eb5468c44ec60e650` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:21.554131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa5865077184c1eb5468c44ec60e650)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:21.556202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:21.623266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:21.623266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:21.635617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdeff0-7907-4cf1-878f-3ef16d7c2ec2 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:21.641932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:21.642634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:21.642954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:21.643242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] PATH_INFO: `/v3/projects/e0eb8bfcee2644b59b890c87e2bee176` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:21.650373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0eb8bfcee2644b59b890c87e2bee176)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:21.653983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:21.815233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa5865077184c1eb5468c44ec60e650'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.815972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa5865077184c1eb5468c44ec60e650'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.819818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0eb8bfcee2644b59b890c87e2bee176', 'request_id': 'req-e0604ede-6d51-4073-ab4e-b839ecb32db9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.823026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa5865077184c1eb5468c44ec60e650'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.823683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa5865077184c1eb5468c44ec60e650'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.827320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0eb8bfcee2644b59b890c87e2bee176', 'request_id': 'req-e0604ede-6d51-4073-ab4e-b839ecb32db9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.937532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa5865077184c1eb5468c44ec60e650'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:21.957190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 963/1959] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:21 2022] DELETE /identity/v3/users/4fa5865077184c1eb5468c44ec60e650 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:21.958119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:21.959882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.001441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0eb8bfcee2644b59b890c87e2bee176', 'request_id': 'req-e0604ede-6d51-4073-ab4e-b839ecb32db9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.013499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] The token cache is being invalidate because project e0eb8bfcee2644b59b890c87e2bee176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:22.013796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.014471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e0604ede-6d51-4073-ab4e-b839ecb32db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.016020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 997/1960] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:21 2022] DELETE /identity/v3/projects/e0eb8bfcee2644b59b890c87e2bee176 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:22.017313 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:22.017616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:22.017895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:22.031378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7cd79f-79b5-4ea7-bea3-1efb040a4606 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:22.036290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:22.037168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:22.037457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:22.037733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] PATH_INFO: `/v3/users/ac677a0592a94d90b469b0a533e12fd1` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:22.038147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:22.038882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac677a0592a94d90b469b0a533e12fd1)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:22.040092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.040823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:22.048549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 998/1961] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:22.049441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:22.050724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.062888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb3ec0-e3fa-4edc-b419-2d89bcc7bb52 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:22.067651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:22.068305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:22.068603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:22.068902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] PATH_INFO: `/v3/users/7c1159270cd0426bab54ad4fbe9e2914` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:22.070246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1159270cd0426bab54ad4fbe9e2914)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:22.072210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:22.099457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:22.217079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac677a0592a94d90b469b0a533e12fd1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.217854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac677a0592a94d90b469b0a533e12fd1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.225309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac677a0592a94d90b469b0a533e12fd1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.226032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac677a0592a94d90b469b0a533e12fd1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.289338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1159270cd0426bab54ad4fbe9e2914'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.290189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1159270cd0426bab54ad4fbe9e2914'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.297194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1159270cd0426bab54ad4fbe9e2914'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.297839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1159270cd0426bab54ad4fbe9e2914'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.355831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac677a0592a94d90b469b0a533e12fd1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.375260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 964/1962] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:21 2022] DELETE /identity/v3/users/ac677a0592a94d90b469b0a533e12fd1 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:22.376733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:22.378633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.424406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ce8529a8-5dbb-4f0c-9e28-270a7a0b43a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1159270cd0426bab54ad4fbe9e2914'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:22.435820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356a8ad-d14c-47fb-9125-b10c18029a4d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:22.439736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:22.440356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:22.440636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:22.440932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:22.442763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.446891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 999/1963] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:22 2022] DELETE /identity/v3/users/7c1159270cd0426bab54ad4fbe9e2914 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:22.452805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:22.453200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:22.453581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:22.475710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:22.477222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.524126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:22.525622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:22.537374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 965/1964] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:22.563773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1000/1965] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:22.574623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:22.576354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:22.588955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313abb8b-3940-422c-975c-347bf2fbc431 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:22.644834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:22.645559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:22.645838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:22.646196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:22.656530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:22.658039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:22.669083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 966/1966] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:22.724730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:23.227389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:23.228970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:23.241337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c749ac-1a8f-4ea0-8bd4-8e0ec2b5cdcf None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:23.246496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:23.247373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:23.247704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:23.248051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] PATH_INFO: `/v3/projects/f9fc36caf1b340aeb08d45d6fcfca194` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:23.249316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9fc36caf1b340aeb08d45d6fcfca194)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:23.251347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:23.328892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:23.330387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:23.351935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef54b-c52b-486f-ac8d-b747f4c4fae1 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:23.361078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:23.361842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:23.362192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:23.362545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] PATH_INFO: `/v3/projects/40aedd1efdbb473bbfa27fe28dc25df1` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:23.364948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=40aedd1efdbb473bbfa27fe28dc25df1)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:23.367385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:23.374963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:23.396014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9fc36caf1b340aeb08d45d6fcfca194', 'request_id': 'req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.407126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9fc36caf1b340aeb08d45d6fcfca194', 'request_id': 'req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.527729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9fc36caf1b340aeb08d45d6fcfca194', 'request_id': 'req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.536838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40aedd1efdbb473bbfa27fe28dc25df1', 'request_id': 'req-b67ffe6b-b52b-4464-957e-c804ae407c2d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.540851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] The token cache is being invalidate because project f9fc36caf1b340aeb08d45d6fcfca194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:23.541259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.542000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.544162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40aedd1efdbb473bbfa27fe28dc25df1', 'request_id': 'req-b67ffe6b-b52b-4464-957e-c804ae407c2d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.547860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1001/1967] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:23 2022] DELETE /identity/v3/projects/f9fc36caf1b340aeb08d45d6fcfca194 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:23.547860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:23.547860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:23.550448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40aedd1efdbb473bbfa27fe28dc25df1', 'request_id': 'req-b67ffe6b-b52b-4464-957e-c804ae407c2d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.619464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e19f1b-8128-48fe-979f-f7dbf7b245ce admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:23.643706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] The token cache is being invalidate because project 40aedd1efdbb473bbfa27fe28dc25df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:23.644046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.644669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b67ffe6b-b52b-4464-957e-c804ae407c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.646235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 967/1968] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:23 2022] DELETE /identity/v3/projects/40aedd1efdbb473bbfa27fe28dc25df1 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:23.649338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:23.649338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:23.649338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:23.670000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:23.671447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:23.685398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:23.685934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:23.686221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:23.686502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] PATH_INFO: `/v3/projects/486e089e13754a0eb90b15f601bc9d8f` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:23.687642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486e089e13754a0eb90b15f601bc9d8f)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:23.689625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:23.736333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 968/1969] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:23.737245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:23.738622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:23.753244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a358a8-3e9e-456a-917b-10b7e268a4f5 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:23.758408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:23.759072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:23.759353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:23.759631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] PATH_INFO: `/v3/users/3ac6c9bd3bf2425c9d0d058d6adf7de1` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:23.760982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac6c9bd3bf2425c9d0d058d6adf7de1)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:23.763148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:23.846068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486e089e13754a0eb90b15f601bc9d8f', 'request_id': 'req-6dd6f7f6-98e9-4427-b1a6-5976670103d7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.860046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486e089e13754a0eb90b15f601bc9d8f', 'request_id': 'req-6dd6f7f6-98e9-4427-b1a6-5976670103d7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.946167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486e089e13754a0eb90b15f601bc9d8f', 'request_id': 'req-6dd6f7f6-98e9-4427-b1a6-5976670103d7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.958060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] The token cache is being invalidate because project 486e089e13754a0eb90b15f601bc9d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:23.958849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.959587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6dd6f7f6-98e9-4427-b1a6-5976670103d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.962298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac6c9bd3bf2425c9d0d058d6adf7de1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.963038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac6c9bd3bf2425c9d0d058d6adf7de1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.973316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac6c9bd3bf2425c9d0d058d6adf7de1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.973581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1002/1970] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:23 2022] DELETE /identity/v3/projects/486e089e13754a0eb90b15f601bc9d8f => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:23.974178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac6c9bd3bf2425c9d0d058d6adf7de1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:23.975347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:23.975623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:23.975898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.005439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:24.006654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:24.041889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac6c9bd3bf2425c9d0d058d6adf7de1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:24.067238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 969/1971] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:23 2022] DELETE /identity/v3/users/3ac6c9bd3bf2425c9d0d058d6adf7de1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:24.100548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1003/1972] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:24.102141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.185944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9088831-1d61-4cc4-8663-349a7c35c3fc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.191195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.191987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.192357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.192704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.193617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:24.194022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.195909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.209770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 970/1973] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:24.214366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.224621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeed46b-be47-4851-a80c-20d93d89f9cc None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.229347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.229954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.230233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.230735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.231750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.233708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.410547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1004/1974] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:24 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:24.415629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.429270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdfa82-26ed-431a-b14f-0a44a3719bcf admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.490919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.491530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.491845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.492191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.492824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.494982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.597888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.599765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:24.626389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1f7a1-f867-491d-b9de-954817f61f57 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.634404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.635278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.635632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.635868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] PATH_INFO: `/v3/projects/1f91058ec6384c9681d02715643517e1` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.638433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f91058ec6384c9681d02715643517e1)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.641662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.648057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 971/1975] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:24 2022] POST /identity/v3/users => generated 291 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:24.652759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.671254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2856-5881-49a2-ae10-77266f1f6b48 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.677523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.678184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.678470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.678726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.679418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.681066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.695574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 972/1976] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:24.700361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.710575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96291d-cc64-47fe-9cb1-d1798b84c5f8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.715551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.716149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.716430 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.716705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.717374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.718966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.729736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 973/1977] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:24.733749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:24.743595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08447f9-a2d0-4729-aa57-fb4e90caaf68 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:24.749173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654a834-efd1-4128-a906-f590e847bb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:24.749816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654a834-efd1-4128-a906-f590e847bb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.750099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654a834-efd1-4128-a906-f590e847bb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.750375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654a834-efd1-4128-a906-f590e847bb50 admin admin] PATH_INFO: `/v3/projects/6fbbfaa558ed4bae8fe12fbb26cd30e2/users/c06deab5f9914b9ea0890616e512a116/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.765819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654a834-efd1-4128-a906-f590e847bb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fbbfaa558ed4bae8fe12fbb26cd30e2, user_id=c06deab5f9914b9ea0890616e512a116, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:24.768143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:24.768619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654a834-efd1-4128-a906-f590e847bb50 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:24.876891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:24.885585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 974/1978] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:24 2022] PUT /identity/v3/projects/6fbbfaa558ed4bae8fe12fbb26cd30e2/users/c06deab5f9914b9ea0890616e512a116/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:24.891522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f91058ec6384c9681d02715643517e1', 'request_id': 'req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:24.897499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:24.898093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:24.898502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:24.898669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f91058ec6384c9681d02715643517e1', 'request_id': 'req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:24.929029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] MFA Rules not processed for user `c06deab5f9914b9ea0890616e512a116`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:24.930258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:25.007637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 975/1979] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:25.007852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f91058ec6384c9681d02715643517e1', 'request_id': 'req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.012625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.026318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] The token cache is being invalidate because project 1f91058ec6384c9681d02715643517e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:25.026580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.027906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.029791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1005/1980] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:24 2022] DELETE /identity/v3/projects/1f91058ec6384c9681d02715643517e1 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:25.034170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.036307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:25.082210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7d7c-29a5-452e-a0c6-0d0138ce4f10 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.087543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.088199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.088481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.088810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.089511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.092465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.120877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6fb2c-cb77-4127-b7bf-860d4191ef0d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.127205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.127205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.127413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.127630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] PATH_INFO: `/v3/users/c0fb3c1b50e84de78d2541e1fe91a712` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.128834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0fb3c1b50e84de78d2541e1fe91a712)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.130807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.208025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 976/1981] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:25 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:25.212230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.213893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:25.277885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b977b0-8fb0-4d22-bc7c-a5993bf47f70 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.282974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.283599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.283890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.284168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.285002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.286915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.347866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0fb3c1b50e84de78d2541e1fe91a712'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.348533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0fb3c1b50e84de78d2541e1fe91a712'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.356240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0fb3c1b50e84de78d2541e1fe91a712'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.356878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0fb3c1b50e84de78d2541e1fe91a712'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.369423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 977/1982] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:25 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:25.374086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.376085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:25.387980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07432446-074d-4ca4-8659-0a9aab394a6e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.392681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.393305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.393588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.393859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.394499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.396083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.407241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 978/1983] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:25.411309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.421902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5bf08-bb6c-41f3-9ed9-56791ac9d8e2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.426681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.427339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.428260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.428260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] PATH_INFO: `/v3/projects/1cf17fef4843461baafb75dafa589ea9/users/c48db476858140f39f5a9a902e5fea58/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.443310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf17fef4843461baafb75dafa589ea9, user_id=c48db476858140f39f5a9a902e5fea58, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.446052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.459145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0fb3c1b50e84de78d2541e1fe91a712'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:25.479713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1006/1984] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:25 2022] DELETE /identity/v3/users/c0fb3c1b50e84de78d2541e1fe91a712 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:25.564080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 979/1985] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:25 2022] PUT /identity/v3/projects/1cf17fef4843461baafb75dafa589ea9/users/c48db476858140f39f5a9a902e5fea58/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:25.568268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.634481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0fb60-5ead-4254-991a-95c53586373d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.694121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14ee20-c528-4981-85ac-e9d26adfaeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.694861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14ee20-c528-4981-85ac-e9d26adfaeda admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.695147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14ee20-c528-4981-85ac-e9d26adfaeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.695515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14ee20-c528-4981-85ac-e9d26adfaeda admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.696163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14ee20-c528-4981-85ac-e9d26adfaeda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.697976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14ee20-c528-4981-85ac-e9d26adfaeda admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.709429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1007/1986] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:25.713601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.724708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159c73a-25cb-4976-8071-db765cf943ab admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:25.729853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:25.731196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.731478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.731754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] PATH_INFO: `/v3/projects/1cf17fef4843461baafb75dafa589ea9/users/c48db476858140f39f5a9a902e5fea58/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.734091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf17fef4843461baafb75dafa589ea9, user_id=c48db476858140f39f5a9a902e5fea58, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:25.737030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:25.750267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 980/1987] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:25 2022] PUT /identity/v3/projects/1cf17fef4843461baafb75dafa589ea9/users/c48db476858140f39f5a9a902e5fea58/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:25.761417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:25.761719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:25.762027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:25.791966 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] MFA Rules not processed for user `c48db476858140f39f5a9a902e5fea58`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:25.793800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:25.901330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1008/1988] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:25.923819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:25.925642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:25.963035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:26.019263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9750480e-8549-49d2-a3e1-d27bbe2385d5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:26.023226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd80925-1773-48f8-ad58-7308fcf59055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:26.023860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd80925-1773-48f8-ad58-7308fcf59055 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.024148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd80925-1773-48f8-ad58-7308fcf59055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.024512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd80925-1773-48f8-ad58-7308fcf59055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.035084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd80925-1773-48f8-ad58-7308fcf59055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:26.036620 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd80925-1773-48f8-ad58-7308fcf59055 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:26.047475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 981/1989] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:26.153107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:26.154707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:26.260519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869355d-a513-44bc-b23e-697af4583d61 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:26.265731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:26.266395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.266723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.267182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] PATH_INFO: `/v3/projects/0567c02ab6514212a9a3e7b25742a40c` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.268376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0567c02ab6514212a9a3e7b25742a40c)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:26.270443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:26.415866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0567c02ab6514212a9a3e7b25742a40c', 'request_id': 'req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.424990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0567c02ab6514212a9a3e7b25742a40c', 'request_id': 'req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.550887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:26.556952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0567c02ab6514212a9a3e7b25742a40c', 'request_id': 'req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.568024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] The token cache is being invalidate because project 0567c02ab6514212a9a3e7b25742a40c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:26.568431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.569193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.571325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1009/1990] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:26 2022] DELETE /identity/v3/projects/0567c02ab6514212a9a3e7b25742a40c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:26.577786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8b08-56ba-44ba-960a-1127393199c0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.578071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8b08-56ba-44ba-960a-1127393199c0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.578408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8b08-56ba-44ba-960a-1127393199c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.599878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dc8c8b08-56ba-44ba-960a-1127393199c0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:26.601181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dc8c8b08-56ba-44ba-960a-1127393199c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:26.660973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:26.662745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:26.669403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 982/1991] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:26.678589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a2a8e-aa4a-40cf-a5ee-52f64b2aaeca admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:26.679013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.679302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.679580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.684379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:26.685069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.685371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.685715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] PATH_INFO: `/v3/users/c06deab5f9914b9ea0890616e512a116` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.687866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06deab5f9914b9ea0890616e512a116)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:26.690046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:26.703901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:26.705323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:26.716957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 983/1992] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:26.719688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:26.729122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb3289-69f2-4e12-ba7d-467e2898e06c None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:26.735382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:26.736060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.736343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.736623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.737458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:26.737736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:26.746181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:26.751982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 984/1993] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:26.759034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:26.767619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7c93b-b0c6-4ede-8b7a-639f52dbf6db admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:26.772562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:26.773210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.773527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.773801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.774437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:26.776396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:26.896586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06deab5f9914b9ea0890616e512a116'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.897405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06deab5f9914b9ea0890616e512a116'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.900937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 985/1994] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:26 2022] POST /identity/v3/projects => generated 361 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:26.905690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:26.905832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06deab5f9914b9ea0890616e512a116'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.906408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06deab5f9914b9ea0890616e512a116'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:26.972703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8707b5-b295-427d-9970-2e59784a9141 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:26.978483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:26.979351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:26.979631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:26.979900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:26.982005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:26.984122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.002463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-107db787-31f0-4b05-b32f-eac53a9ef0b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06deab5f9914b9ea0890616e512a116'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.021964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1010/1995] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:26 2022] DELETE /identity/v3/users/c06deab5f9914b9ea0890616e512a116 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:27.026389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.026681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.026959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.047217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:27.048455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:27.090636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 986/1996] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:26 2022] POST /identity/v3/users => generated 281 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:27.095570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.098832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:27.124899 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1011/1997] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:27.130898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649cee0-4a1e-400d-b354-0c68c656d3ec admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.135324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.136449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.137825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.138107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.138385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.138717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:27.139196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.140799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.152208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 987/1998] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:27.156090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.166577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f21f0-d4e0-430a-bd26-788c23cfffec admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.173137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.173862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.174216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.174490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.175164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.177000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.188955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 988/1999] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:27.193293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.203833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd3681-abb6-455f-9814-b196e38580c3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.209115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.209800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.210252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.210778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] PATH_INFO: `/v3/projects/a9cc6ce001f745d6a8d816935890be1a/users/50b6b95fd5a048fe90be652e425b2444/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.212723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bd7fd-47be-4ab2-9e17-bb5709b23128 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.224314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765170e-f7cb-46fc-97d2-f2901a34a1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.224314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765170e-f7cb-46fc-97d2-f2901a34a1ca service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.224314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765170e-f7cb-46fc-97d2-f2901a34a1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.224314 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765170e-f7cb-46fc-97d2-f2901a34a1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.226350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9cc6ce001f745d6a8d816935890be1a, user_id=50b6b95fd5a048fe90be652e425b2444, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.229074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.229938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765170e-f7cb-46fc-97d2-f2901a34a1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.232616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765170e-f7cb-46fc-97d2-f2901a34a1ca service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.243009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1012/2000] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:27.322526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 989/2001] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:27 2022] PUT /identity/v3/projects/a9cc6ce001f745d6a8d816935890be1a/users/50b6b95fd5a048fe90be652e425b2444/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:27.335186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.335503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.335866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.367186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] MFA Rules not processed for user `50b6b95fd5a048fe90be652e425b2444`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:27.368420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:27.436497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1013/2002] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:27.488028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.488028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:27.535691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf04f93-af2b-40b9-a671-9717d8ba4910 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.539584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:27.539888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.540604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.541042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.541376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.550501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.553744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.563988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 990/2003] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:27.580400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.644070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be1494-03cd-4363-9ff4-1ce7b079db2f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.649173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.649847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.650131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.650410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] PATH_INFO: `/v3/users/50b6b95fd5a048fe90be652e425b2444` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.651629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b6b95fd5a048fe90be652e425b2444)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.653615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.702173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:27.703599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:27.718351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd47b6-ae0c-447e-bb4d-5b9c78bbd271 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:27.723528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:27.724170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:27.725072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:27.725443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] PATH_INFO: `/v3/projects/6fbbfaa558ed4bae8fe12fbb26cd30e2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:27.727645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fbbfaa558ed4bae8fe12fbb26cd30e2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:27.729044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:27.970903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b6b95fd5a048fe90be652e425b2444'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.971638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b6b95fd5a048fe90be652e425b2444'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.974863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fbbfaa558ed4bae8fe12fbb26cd30e2', 'request_id': 'req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.979611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b6b95fd5a048fe90be652e425b2444'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.980906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b6b95fd5a048fe90be652e425b2444'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.983191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fbbfaa558ed4bae8fe12fbb26cd30e2', 'request_id': 'req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:27.990251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fbbfaa558ed4bae8fe12fbb26cd30e2', 'request_id': 'req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.087510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f2065dd6-53ca-4e88-9b66-6da05256464c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b6b95fd5a048fe90be652e425b2444'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.106410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1014/2004] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:27 2022] DELETE /identity/v3/users/50b6b95fd5a048fe90be652e425b2444 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:28.110383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:28.110695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:28.110989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:28.130652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:28.131982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:28.194158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] The token cache is being invalidate because project 6fbbfaa558ed4bae8fe12fbb26cd30e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:28.194158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.195587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.197773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 991/2005] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:27 2022] DELETE /identity/v3/projects/6fbbfaa558ed4bae8fe12fbb26cd30e2 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:28.202165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:28.204047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:28.227814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1015/2006] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:28.252203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:28.254481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:28.302556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:28.308472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ad2c8-e5b5-4078-bb0a-12c09d6e7173 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:28.314741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:28.315395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:28.315832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:28.316263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] PATH_INFO: `/v3/users/c48db476858140f39f5a9a902e5fea58` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:28.317631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48db476858140f39f5a9a902e5fea58)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:28.319593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:28.336784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21d00d-a9e8-4419-87b8-b00199112a90 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:28.341286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:28.341831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:28.342222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:28.342508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:28.352439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:28.354032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:28.365978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1016/2007] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:28.564215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48db476858140f39f5a9a902e5fea58'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.565299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48db476858140f39f5a9a902e5fea58'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.573496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48db476858140f39f5a9a902e5fea58'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.574776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48db476858140f39f5a9a902e5fea58'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.762546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e417fd66-9409-4f75-b567-2d7420c0b354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48db476858140f39f5a9a902e5fea58'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:28.782337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 992/2008] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:28 2022] DELETE /identity/v3/users/c48db476858140f39f5a9a902e5fea58 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:28.896321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:28.899211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:28.922385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37315797-93c5-468d-bac2-497259db9d1b service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:28.993373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:28.994556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:28.994556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:28.994750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] PATH_INFO: `/v3/projects/a9cc6ce001f745d6a8d816935890be1a` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:28.996410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9cc6ce001f745d6a8d816935890be1a)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:28.998394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.161592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9cc6ce001f745d6a8d816935890be1a', 'request_id': 'req-67004e93-50cf-43ed-b40c-511414a3d465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.170586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9cc6ce001f745d6a8d816935890be1a', 'request_id': 'req-67004e93-50cf-43ed-b40c-511414a3d465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.356105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9cc6ce001f745d6a8d816935890be1a', 'request_id': 'req-67004e93-50cf-43ed-b40c-511414a3d465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.368651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] The token cache is being invalidate because project a9cc6ce001f745d6a8d816935890be1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:29.368975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.369623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.371531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1017/2009] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:28 2022] DELETE /identity/v3/projects/a9cc6ce001f745d6a8d816935890be1a => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:29.376736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a8fa3-0273-44f6-95d6-766e5b3bef89 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.377161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a8fa3-0273-44f6-95d6-766e5b3bef89 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.377435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a8fa3-0273-44f6-95d6-766e5b3bef89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.399294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-707a8fa3-0273-44f6-95d6-766e5b3bef89 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:29.400528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-707a8fa3-0273-44f6-95d6-766e5b3bef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:29.412772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.414531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:29.477524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 993/2010] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:29.490382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.490680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.490931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.502025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67004e93-50cf-43ed-b40c-511414a3d465 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.507357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.508005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.508288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.508570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] PATH_INFO: `/v3/projects/1cf17fef4843461baafb75dafa589ea9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.509905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf17fef4843461baafb75dafa589ea9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.512793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.514532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:29.515790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:29.524914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 994/2011] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:29.532008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.541807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4383ab-b17e-4ae7-9005-4813489b388a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.547361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.548879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.549186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.549461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.550329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:29.550632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.552312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.564578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 995/2012] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:29.569366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.584493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b081a8-5f42-485e-a56e-8dc6d642e85a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.592455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.593312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.593593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.593869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.594753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.596909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.646201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf17fef4843461baafb75dafa589ea9', 'request_id': 'req-20bfff11-b91f-440f-99ac-78babcfc80b7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.654623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf17fef4843461baafb75dafa589ea9', 'request_id': 'req-20bfff11-b91f-440f-99ac-78babcfc80b7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.658446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:29.692992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 996/2013] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:29.697938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.764267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc07e6-8574-4fec-b7ab-2d7c3e50196a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.768974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.769609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.769877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.770153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.770806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.772719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.800654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf17fef4843461baafb75dafa589ea9', 'request_id': 'req-20bfff11-b91f-440f-99ac-78babcfc80b7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.812207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] The token cache is being invalidate because project 1cf17fef4843461baafb75dafa589ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:29.812507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.813184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-20bfff11-b91f-440f-99ac-78babcfc80b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:29.814698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1018/2014] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:29 2022] DELETE /identity/v3/projects/1cf17fef4843461baafb75dafa589ea9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:29.819370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283075-c1df-43f0-bcda-3eff149b162e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.819652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283075-c1df-43f0-bcda-3eff149b162e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.819929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92283075-c1df-43f0-bcda-3eff149b162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.840492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-92283075-c1df-43f0-bcda-3eff149b162e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:29.841750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-92283075-c1df-43f0-bcda-3eff149b162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:29.850792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1019/2015] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:29.860372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.860565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.860979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.882618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:29.883817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:29.900146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 997/2016] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:29.901021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1020/2017] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:29.903829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.905439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.905875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:29.915713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7867fca-bddb-440b-b36d-bdf879d64dbc None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.927415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.927640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dde3fc-3cf7-460d-84e8-bcefe3e40eb9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.928459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.929487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.929985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.931242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:29.931792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.933023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.934400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.934988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.935200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.935651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.936527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.938518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.953871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 998/2018] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:29.957003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1021/2019] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:29.963947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.965551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:29.975934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e6250-6aa3-4ef9-b848-71b1a425cba5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.981476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.981965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.982246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.982553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.983218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:29.984260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c4821-b1d6-422a-a2be-1c4b119e6f2b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:29.985187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:29.993247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:29.994441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:29.994961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:29.996493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:29.997818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.000644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.015059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 999/2020] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:30.019420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.030526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e59f2-1839-4cfb-a63f-d6a45736ee83 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.036217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36387f-ab88-4576-b582-94eedeeb34d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.036937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36387f-ab88-4576-b582-94eedeeb34d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.037427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36387f-ab88-4576-b582-94eedeeb34d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.037704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36387f-ab88-4576-b582-94eedeeb34d7 admin admin] PATH_INFO: `/v3/projects/35e878b31d0d484eaec8d95425311448/users/72faa8dd0c814be0b477991aa73dba50/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.055910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36387f-ab88-4576-b582-94eedeeb34d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e878b31d0d484eaec8d95425311448, user_id=72faa8dd0c814be0b477991aa73dba50, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.059405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36387f-ab88-4576-b582-94eedeeb34d7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.112118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1022/2021] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:29 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:30.112713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.175712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1000/2022] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:30 2022] PUT /identity/v3/projects/35e878b31d0d484eaec8d95425311448/users/72faa8dd0c814be0b477991aa73dba50/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:30.184024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef444e-c5fb-4f6e-8c13-d696748a3304 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.187896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.188376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.188817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.222843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] MFA Rules not processed for user `72faa8dd0c814be0b477991aa73dba50`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:30.224124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:30.260287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.261070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.261419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.262202 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.263449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.265515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.271136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:30.307758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1001/2023] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:30.321606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.322993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:30.369032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1023/2024] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:30 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:30.380195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.393288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9154c2-f8d4-4ec8-8b71-beffd4058ab4 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.394821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a4f7-cc56-46e4-996a-788f4224fc17 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.399064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.399930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.400271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.400552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.400823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.401498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.401988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.403115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.404007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.406011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.411521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.413092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.421534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1024/2025] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:30.427542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1002/2026] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:30.433997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.447636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34743e5-8087-4aca-a6d9-31008deced09 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.457840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82c861-dfb9-43b5-94fc-559a3a20d49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.459590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82c861-dfb9-43b5-94fc-559a3a20d49e admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.459874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82c861-dfb9-43b5-94fc-559a3a20d49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.460575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82c861-dfb9-43b5-94fc-559a3a20d49e admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.462384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82c861-dfb9-43b5-94fc-559a3a20d49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.467869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82c861-dfb9-43b5-94fc-559a3a20d49e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.492788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1025/2027] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:30.497400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.508757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b8acc-1e49-400a-9d34-429a95674137 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.514646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.515386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.515650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.516077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] PATH_INFO: `/v3/projects/5eb0c8cd9f2f4f09a512096893244c97/users/f6c2e1f29e6847a3a15deee5209b21cb/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.536431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb0c8cd9f2f4f09a512096893244c97, user_id=f6c2e1f29e6847a3a15deee5209b21cb, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.539623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.636075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1003/2028] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:30 2022] PUT /identity/v3/projects/5eb0c8cd9f2f4f09a512096893244c97/users/f6c2e1f29e6847a3a15deee5209b21cb/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:30.646968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335d9e8-c0c3-4b5f-a047-9171351ba1cc None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.647241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335d9e8-c0c3-4b5f-a047-9171351ba1cc None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.647520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335d9e8-c0c3-4b5f-a047-9171351ba1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.679670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7335d9e8-c0c3-4b5f-a047-9171351ba1cc None None] MFA Rules not processed for user `f6c2e1f29e6847a3a15deee5209b21cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:30.680887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7335d9e8-c0c3-4b5f-a047-9171351ba1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:30.762367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1026/2029] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:30.773603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:30.775414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:30.868479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab97b75-ca0c-45cd-b307-015ff43a00a0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:30.880812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c0d64-8f80-44d6-87c1-04e225cf3e34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:30.881497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c0d64-8f80-44d6-87c1-04e225cf3e34 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:30.881781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c0d64-8f80-44d6-87c1-04e225cf3e34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:30.882054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c0d64-8f80-44d6-87c1-04e225cf3e34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:30.899363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c0d64-8f80-44d6-87c1-04e225cf3e34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:30.901198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c0d64-8f80-44d6-87c1-04e225cf3e34 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:30.915997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1004/2030] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:31.053531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:31.928012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:32.934707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1688336-89b1-4883-98e1-99b642a58165 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:32.935190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1688336-89b1-4883-98e1-99b642a58165 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:32.935458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1688336-89b1-4883-98e1-99b642a58165 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:32.936841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1027/2031] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:36:32.950147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95e81e-3ba0-416e-855a-8175223f36b8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:32.950674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95e81e-3ba0-416e-855a-8175223f36b8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:32.950771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95e81e-3ba0-416e-855a-8175223f36b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:32.973743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bf95e81e-3ba0-416e-855a-8175223f36b8 None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:32.975073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bf95e81e-3ba0-416e-855a-8175223f36b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:33.055123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1005/2032] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:36:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:33.069766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcdb5-9eac-4086-b0c1-8da4e3cf202b None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:33.070270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcdb5-9eac-4086-b0c1-8da4e3cf202b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:33.070684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efcdb5-9eac-4086-b0c1-8da4e3cf202b None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:33.072268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1028/2033] 10.222.0.42 () {54 vars in 961 bytes} [Tue Oct 4 10:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:36:33.077961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:33.078258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:33.078567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:33.099553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:33.101134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:33.112289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1006/2034] 10.222.0.42 () {58 vars in 1058 bytes} [Tue Oct 4 10:36:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:33.119948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efcdb5-9eac-4086-b0c1-8da4e3cf202b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:33.134608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efcdb5-9eac-4086-b0c1-8da4e3cf202b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:33.145172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:33.145172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:33.145172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:33.145172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:33.151355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:33.153613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:33.161756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:33.163467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:33.174655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1029/2035] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:33.223637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e929c-566d-48a2-86dd-d2eeb539bd5b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:33.228353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:33.229018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:33.229303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:33.229573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] PATH_INFO: `/v3/users/7b69eb143b5d42c3b4a8cb292c6a70d7` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:33.231422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b69eb143b5d42c3b4a8cb292c6a70d7)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:33.233381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:33.491656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b69eb143b5d42c3b4a8cb292c6a70d7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:33.492277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b69eb143b5d42c3b4a8cb292c6a70d7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:33.500421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b69eb143b5d42c3b4a8cb292c6a70d7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:33.501102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b69eb143b5d42c3b4a8cb292c6a70d7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:33.528674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:33.591959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:33.592702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b69eb143b5d42c3b4a8cb292c6a70d7'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:33.611594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1007/2036] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:33 2022] DELETE /identity/v3/users/7b69eb143b5d42c3b4a8cb292c6a70d7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:33.921908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:33.941435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3244932-fcb1-4915-bcd6-d1b6bc2f97e6 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.036881 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.037858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.038123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.038402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] PATH_INFO: `/v3/projects/2e7330488d40469199873490c5fd2bb4` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.041177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7330488d40469199873490c5fd2bb4)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.043688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.056939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:34.059617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:34.076448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21aa23d-193f-4f0f-bcb5-6b68cee8f76f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.082483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.083164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.083942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.084642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] PATH_INFO: `/v3/users/72faa8dd0c814be0b477991aa73dba50` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.086443 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72faa8dd0c814be0b477991aa73dba50)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.088594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.181285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7330488d40469199873490c5fd2bb4', 'request_id': 'req-3533b8fe-55b6-47b4-a81b-2774be0583ed'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.188208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7330488d40469199873490c5fd2bb4', 'request_id': 'req-3533b8fe-55b6-47b4-a81b-2774be0583ed'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.196467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:34.270371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7330488d40469199873490c5fd2bb4', 'request_id': 'req-3533b8fe-55b6-47b4-a81b-2774be0583ed'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.286286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] The token cache is being invalidate because project 2e7330488d40469199873490c5fd2bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:34.286286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.286286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3533b8fe-55b6-47b4-a81b-2774be0583ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.286286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1030/2037] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:33 2022] DELETE /identity/v3/projects/2e7330488d40469199873490c5fd2bb4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:34.289940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736058c-a298-48e2-ac69-212282f11d6c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.290221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736058c-a298-48e2-ac69-212282f11d6c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.290494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736058c-a298-48e2-ac69-212282f11d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.311680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7736058c-a298-48e2-ac69-212282f11d6c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:34.312647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7736058c-a298-48e2-ac69-212282f11d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:34.324945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72faa8dd0c814be0b477991aa73dba50'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.325603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72faa8dd0c814be0b477991aa73dba50'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.332606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72faa8dd0c814be0b477991aa73dba50'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.333486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72faa8dd0c814be0b477991aa73dba50'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.374889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1031/2038] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:34.382833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.383251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.383488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.405022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:34.406330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:34.414834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1032/2039] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:34.419930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:34.430102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e856c0-0ea9-4e9f-a6bf-eff45dd4a5a7 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.435221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.435867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.438418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.438418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.438418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:34.438418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.439232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.452356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1033/2040] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:34.456720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:34.467580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa384c-816d-4b84-853a-486987faed98 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.472291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.472916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.473246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.473474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.474111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.476031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.527886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-04092c59-ce03-4c6d-9455-f10b47ce6567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72faa8dd0c814be0b477991aa73dba50'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.549261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1008/2041] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:34 2022] DELETE /identity/v3/users/72faa8dd0c814be0b477991aa73dba50 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:34.553665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.553945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.554216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.579236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:34.580476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:34.663693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1009/2042] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:34.674951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:34.676351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:34.713424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1034/2043] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:34 2022] POST /identity/v3/projects => generated 371 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:34.717621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:34.737146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabea82-f5ee-456e-96c2-0aa3b343150f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.793684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e06e8-83a7-46bd-970b-61ed87e01899 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.798778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.799640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.800206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.800560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.800787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.801480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.801770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.802034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.802134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.804225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.813824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.815829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.828323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1010/2044] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:34.989418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1035/2045] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:34 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:34.998579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.008893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a9bc0-2a49-4be7-b045-4329d3341753 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.013851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.014454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.014757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.015027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.015661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.017309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.027784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1011/2046] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:35.032308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.042651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646af03-1703-4d3f-82cc-c67825ef6574 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.047524 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed662277-f626-4a2f-a429-a97224f008b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.048068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed662277-f626-4a2f-a429-a97224f008b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.048351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed662277-f626-4a2f-a429-a97224f008b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.048628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed662277-f626-4a2f-a429-a97224f008b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.049308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed662277-f626-4a2f-a429-a97224f008b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.050925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed662277-f626-4a2f-a429-a97224f008b4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.061462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1036/2047] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:35.066032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.077023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbfded-910e-42db-afcb-2e90acf1860e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.082507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.087186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.087186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.087186 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] PATH_INFO: `/v3/projects/a027b44d2783413ba8ed033b81db4920/users/5d09dd5e28af4dea9a744247449311ae/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.101044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a027b44d2783413ba8ed033b81db4920, user_id=5d09dd5e28af4dea9a744247449311ae, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.103728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.191269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1012/2048] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:35 2022] PUT /identity/v3/projects/a027b44d2783413ba8ed033b81db4920/users/5d09dd5e28af4dea9a744247449311ae/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:35.203082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.203384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.203757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.236791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] MFA Rules not processed for user `5d09dd5e28af4dea9a744247449311ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:35.238364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.306768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1037/2049] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:35.321049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.381472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.383375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.384196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61be69-1117-4a79-ad6c-62bbd7f8afc3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.391508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.392132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.392482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.392675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.393655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.399540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f984e-20df-44cf-a339-dd3bef7ef5bb None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.401482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.403930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:35.408402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.409458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.411897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.411897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] PATH_INFO: `/v3/projects/35e878b31d0d484eaec8d95425311448` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.411897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e878b31d0d484eaec8d95425311448)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.413315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.504909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:35.540446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1013/2050] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:35.545271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.551767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e878b31d0d484eaec8d95425311448', 'request_id': 'req-41f57c87-01ca-45cd-a157-abb8bb2fbda1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.558014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e878b31d0d484eaec8d95425311448', 'request_id': 'req-41f57c87-01ca-45cd-a157-abb8bb2fbda1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.608535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698de51c-f833-47a0-ae19-003c3b5ccc2e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.613401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.614031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.614300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.614736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.615383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.617349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.635501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e878b31d0d484eaec8d95425311448', 'request_id': 'req-41f57c87-01ca-45cd-a157-abb8bb2fbda1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.647901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] The token cache is being invalidate because project 35e878b31d0d484eaec8d95425311448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:35.648161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.648833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-41f57c87-01ca-45cd-a157-abb8bb2fbda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.650601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1038/2051] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:35 2022] DELETE /identity/v3/projects/35e878b31d0d484eaec8d95425311448 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:35.655925 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9f04-dd70-4b90-806d-2cfd6a9ff35f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.656218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9f04-dd70-4b90-806d-2cfd6a9ff35f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.656492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b9f04-dd70-4b90-806d-2cfd6a9ff35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.677295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8c0b9f04-dd70-4b90-806d-2cfd6a9ff35f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:35.678517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8c0b9f04-dd70-4b90-806d-2cfd6a9ff35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.687766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1039/2052] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:35.695507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.695798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.696073 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.718741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:35.720149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.725541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1014/2053] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:35.726633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.728530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.734194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1040/2054] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:35.734194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.743097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfe5c1-1248-47a1-bd07-1ceb6578b924 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.748147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b51c8-27b5-4d37-a00d-51917cf4a1ac admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.750575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.751701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.752081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.752435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.753249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:35.753594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.755652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.756521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.757325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.757712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.758053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.758960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.760897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.777778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1041/2055] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:36:35.777976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1015/2056] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:35.778483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.779166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.788409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625ab0d-5652-4872-868b-f7f827efd0a9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.789227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e0873-c123-440e-8932-74f82c0bb8ec admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.793335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.793968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.797079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.797079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.797079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.797322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.797322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.797322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.797322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] PATH_INFO: `/v3/projects/fddaaa99777d42f4b77ec1ade3fc3c81/users/6d3e9f82720e4f2d93620b79f8e79ef1/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.797650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.812287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddaaa99777d42f4b77ec1ade3fc3c81, user_id=6d3e9f82720e4f2d93620b79f8e79ef1, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.815277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.818880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:35.889401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1042/2057] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:35.893562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.910825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:35.918685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1016/2058] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:35 2022] PUT /identity/v3/projects/fddaaa99777d42f4b77ec1ade3fc3c81/users/6d3e9f82720e4f2d93620b79f8e79ef1/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:35.922483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.963666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c87db8-97db-47ca-b863-e308165ea90f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.987852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e161-b06b-4bb0-ace1-5142cf65f00c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.992560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.993245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.993528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.993799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.994438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.996611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.007128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1017/2059] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:36.012444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.022022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d96c6e-b529-46c7-9607-71c065fe7e8c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.023040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.023663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.024058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.024344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.025013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.026986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.031699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57560a-53da-427f-b9c4-583f46b5030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.040771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57560a-53da-427f-b9c4-583f46b5030d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.040771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57560a-53da-427f-b9c4-583f46b5030d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.040771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57560a-53da-427f-b9c4-583f46b5030d admin admin] PATH_INFO: `/v3/projects/fddaaa99777d42f4b77ec1ade3fc3c81/users/6d3e9f82720e4f2d93620b79f8e79ef1/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.040771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57560a-53da-427f-b9c4-583f46b5030d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fddaaa99777d42f4b77ec1ade3fc3c81, user_id=6d3e9f82720e4f2d93620b79f8e79ef1, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.040771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df57560a-53da-427f-b9c4-583f46b5030d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.048715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1018/2060] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:36 2022] PUT /identity/v3/projects/fddaaa99777d42f4b77ec1ade3fc3c81/users/6d3e9f82720e4f2d93620b79f8e79ef1/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:36.067611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.068265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.068265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.076786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.099511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] MFA Rules not processed for user `6d3e9f82720e4f2d93620b79f8e79ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:36.100781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.144796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1043/2061] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:36.148744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.159566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6fe88-288f-4a13-b6d4-17b0b8d329af admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.175160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1019/2062] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:36.185890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.187888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.189511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.218885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.219522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.219788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.220061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.220709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.222538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.232543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1044/2063] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:36.236549 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.246456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecde834-e4ab-4867-aadd-a6d35ce182f0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.251231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4905b-ce7b-4f5c-921d-39839f31051b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.251738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.252420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.252774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.253144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.253850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.259656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3be4ac-718b-4614-b13a-99eaf4a46d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.259919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.260016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3be4ac-718b-4614-b13a-99eaf4a46d25 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.260016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3be4ac-718b-4614-b13a-99eaf4a46d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.260016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3be4ac-718b-4614-b13a-99eaf4a46d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.265993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1045/2064] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:36.267205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3be4ac-718b-4614-b13a-99eaf4a46d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.268781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3be4ac-718b-4614-b13a-99eaf4a46d25 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.270978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.280670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1020/2065] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:36.286973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc79aec-0bad-4c50-a8d7-e186854b7f3b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.291536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.292215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.292560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.292841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] PATH_INFO: `/v3/projects/9cd88d8702cd493d956ee8d024362b40/users/26b1983b7d57440ea3de52af917fbb94/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.313215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd88d8702cd493d956ee8d024362b40, user_id=26b1983b7d57440ea3de52af917fbb94, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.316005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.322420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.454783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.460802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1046/2066] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:36 2022] PUT /identity/v3/projects/9cd88d8702cd493d956ee8d024362b40/users/26b1983b7d57440ea3de52af917fbb94/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:36.472075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.472389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.472616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.500647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] MFA Rules not processed for user `26b1983b7d57440ea3de52af917fbb94`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:36.501913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.533405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.569970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1021/2067] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:36.581662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.583880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.594178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b36882-69da-4e70-97a1-3d7d0c5e3356 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.598249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b84f7-2383-4788-9b5e-583bb7dcc9ee None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.598772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.599414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.599699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.599866 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.600131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.600821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.601135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.601405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.602149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.611208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.612645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.623894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1022/2068] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:36.670907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.672368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dd8b9-6c94-44a2-89c5-d9ce9396fd9c service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.681505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1047/2069] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:36.699833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.727698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:37.642066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:37.699883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:38.207501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:38.345095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:38.658867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:39.253427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:39.790733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:40.036540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:41.018793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:41.321675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:42.750676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:43.110090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:43.601725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:43.853181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:43.950280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.018661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.213239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.241792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.533099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.636572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.778998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.813187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:44.898508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:45.075231 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:45.270134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:45.585142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:45.790425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:46.035632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:47.363182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:47.364724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:47.435551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03f8d2-66cd-4d6f-9e1a-c32173014c20 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:47.441741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:47.442445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:47.442817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:47.443188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] PATH_INFO: `/v3/users/f6c2e1f29e6847a3a15deee5209b21cb` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:47.444670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c2e1f29e6847a3a15deee5209b21cb)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:47.447263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:47.681292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c2e1f29e6847a3a15deee5209b21cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:47.682078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c2e1f29e6847a3a15deee5209b21cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:47.691319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c2e1f29e6847a3a15deee5209b21cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:47.696179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c2e1f29e6847a3a15deee5209b21cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:47.791501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c2e1f29e6847a3a15deee5209b21cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:47.812664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1023/2070] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:47 2022] DELETE /identity/v3/users/f6c2e1f29e6847a3a15deee5209b21cb => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:47.817174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:47.817689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:47.817789 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:47.837113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:47.838333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:47.918635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1048/2071] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:47.928248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:47.930103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:48.001820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c3c81-c74c-4fe8-8791-806b1eff4c2d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:48.005768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bbf90-4ac5-4f20-8d68-fae8d1ee6468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:48.006463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bbf90-4ac5-4f20-8d68-fae8d1ee6468 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:48.006817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bbf90-4ac5-4f20-8d68-fae8d1ee6468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:48.007692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bbf90-4ac5-4f20-8d68-fae8d1ee6468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:48.017107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bbf90-4ac5-4f20-8d68-fae8d1ee6468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:48.018657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bbf90-4ac5-4f20-8d68-fae8d1ee6468 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:48.037266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1024/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:48.531500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:48.533351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:48.548685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c370e-67a9-4a41-9b29-2d1468031720 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:48.553859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:48.554578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:48.554921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:48.555266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] PATH_INFO: `/v3/projects/5eb0c8cd9f2f4f09a512096893244c97` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:48.556567 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb0c8cd9f2f4f09a512096893244c97)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:48.559304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:48.723137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb0c8cd9f2f4f09a512096893244c97', 'request_id': 'req-95e4ff6a-071a-4e57-a307-aeba065e02d8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:48.729386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb0c8cd9f2f4f09a512096893244c97', 'request_id': 'req-95e4ff6a-071a-4e57-a307-aeba065e02d8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:48.828498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb0c8cd9f2f4f09a512096893244c97', 'request_id': 'req-95e4ff6a-071a-4e57-a307-aeba065e02d8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:48.845373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] The token cache is being invalidate because project 5eb0c8cd9f2f4f09a512096893244c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:48.845829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:48.846676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-95e4ff6a-071a-4e57-a307-aeba065e02d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:48.848428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1049/2073] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:48 2022] DELETE /identity/v3/projects/5eb0c8cd9f2f4f09a512096893244c97 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:48.853272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:48.853703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:48.853818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:48.887532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:48.894172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:48.904325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:48.982687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1025/2074] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:48.990677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:48.991147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:48.991422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.013732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:49.014991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:49.025114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1050/2075] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:49.036498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.046400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df08126-30a5-4b8f-a57a-eca7bbdc9284 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.051212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.055469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.055469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.055469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.055469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:49.055469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.055469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.068967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1026/2076] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:49.071439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.081777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93df837-c2dd-4ba2-b358-2c03d9ce623b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.086747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.087355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.087632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.087906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.088535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.091114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.208379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1051/2077] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:49 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:49.213241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.223862 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48425db-9123-4f27-b782-14b756d5f0fe admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.281208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.281826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.282116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.282393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.283031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.285001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.384330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1027/2078] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:49 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:49.389090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.399760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e679676-bee4-45a1-a385-dd84a98e1cd6 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.405186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.405793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.406072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.406348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.406987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.408585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.419047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1052/2079] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:49.423257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.434289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b0ac4-22e2-498f-ab8b-50cea7194b6e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.439094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29cee9-c62f-4769-81c3-f4317af1386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.439740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29cee9-c62f-4769-81c3-f4317af1386e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.440051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29cee9-c62f-4769-81c3-f4317af1386e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.440319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29cee9-c62f-4769-81c3-f4317af1386e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.440997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29cee9-c62f-4769-81c3-f4317af1386e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.442585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29cee9-c62f-4769-81c3-f4317af1386e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.454237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1028/2080] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:49.458214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.468751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7e606-f0db-44ab-8c6a-99eca44a7444 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.473760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.474448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.474728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.475004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] PATH_INFO: `/v3/projects/4a52bd921146485fb107be1607f9b7ed/users/4ae07662def846e399f4dea29d71f58c/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.490110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a52bd921146485fb107be1607f9b7ed, user_id=4ae07662def846e399f4dea29d71f58c, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.492763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.596287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1053/2081] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:49 2022] PUT /identity/v3/projects/4a52bd921146485fb107be1607f9b7ed/users/4ae07662def846e399f4dea29d71f58c/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:49.609312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.609595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.609864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.640515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] MFA Rules not processed for user `4ae07662def846e399f4dea29d71f58c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:49.641890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:49.706978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1029/2082] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:49.711784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.725630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:49.775616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c856d19-7019-4e15-9c68-57c1ff0dd858 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.781466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.782078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.782358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.782652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.783267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.785244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:49.907836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1054/2083] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:49 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:49.911969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:49.918984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:49.923370 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1a09-cfd1-42d2-b8c6-fda628c67064 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:49.992005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:49.992717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:49.993126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:49.993450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:49.994262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:49.997048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:50.099993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1030/2084] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:49 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:50.102200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:50.115079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8758d-ff57-47fe-ab3c-b6dff9cdd0c7 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:50.120612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:50.121490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:50.121906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:50.122212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:50.122905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:50.124539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:50.137909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1055/2085] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:50.142143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:50.152505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f5459-c5fa-4803-8b94-53db501a0273 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:50.157396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:50.158053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:50.158332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:50.158605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] PATH_INFO: `/v3/projects/65cb532e7b7e40e98947d46b7e374e1d/users/97b29f169f344bec862f29352b59f236/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:50.174651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cb532e7b7e40e98947d46b7e374e1d, user_id=97b29f169f344bec862f29352b59f236, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:50.177389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:50.304828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1031/2086] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:50 2022] PUT /identity/v3/projects/65cb532e7b7e40e98947d46b7e374e1d/users/97b29f169f344bec862f29352b59f236/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:50.309519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:50.320080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17da90-506f-4de5-83b6-9c17104620b2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:50.380059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f37aca-33fd-41eb-a214-ab8133720848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:50.380663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f37aca-33fd-41eb-a214-ab8133720848 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:50.380975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f37aca-33fd-41eb-a214-ab8133720848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:50.381256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f37aca-33fd-41eb-a214-ab8133720848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:50.381898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f37aca-33fd-41eb-a214-ab8133720848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:50.383504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f37aca-33fd-41eb-a214-ab8133720848 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:50.393340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1056/2087] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:50.397571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:50.408664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045b677-1de8-4d2e-95a6-8f739d95ba07 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:50.413923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:50.414618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:50.414903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:50.415177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] PATH_INFO: `/v3/projects/65cb532e7b7e40e98947d46b7e374e1d/users/97b29f169f344bec862f29352b59f236/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:50.418137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65cb532e7b7e40e98947d46b7e374e1d, user_id=97b29f169f344bec862f29352b59f236, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:50.420487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:50.431336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1032/2088] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:50 2022] PUT /identity/v3/projects/65cb532e7b7e40e98947d46b7e374e1d/users/97b29f169f344bec862f29352b59f236/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:50.442047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:50.442336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:50.442639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:50.471711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] MFA Rules not processed for user `97b29f169f344bec862f29352b59f236`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:50.473000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:50.545936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1057/2089] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:50.557092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:50.559910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:50.623162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f43811-ed9f-4da1-88a5-65631f1d5578 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:50.626868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a95a3-a9e8-44a2-aff5-9a7e27e2642a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:50.627475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a95a3-a9e8-44a2-aff5-9a7e27e2642a service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:50.627784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a95a3-a9e8-44a2-aff5-9a7e27e2642a service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:50.628245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a95a3-a9e8-44a2-aff5-9a7e27e2642a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:50.691255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a95a3-a9e8-44a2-aff5-9a7e27e2642a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:50.693302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a95a3-a9e8-44a2-aff5-9a7e27e2642a service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:50.703412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1033/2090] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:53.821464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:53.824089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:53.835153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:53.892225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656a9d7-ff39-42c3-99f1-4fa7bf20bf6e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:53.897448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:53.898082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:53.899103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:53.899103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] PATH_INFO: `/v3/users/26b1983b7d57440ea3de52af917fbb94` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:53.899748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b1983b7d57440ea3de52af917fbb94)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:53.902037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:54.095050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b1983b7d57440ea3de52af917fbb94'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.095692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b1983b7d57440ea3de52af917fbb94'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.102882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b1983b7d57440ea3de52af917fbb94'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.103712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b1983b7d57440ea3de52af917fbb94'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.184691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b1983b7d57440ea3de52af917fbb94'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.209237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1058/2091] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:53 2022] DELETE /identity/v3/users/26b1983b7d57440ea3de52af917fbb94 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:54.213830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:54.214343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:54.214456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:54.234018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:54.235228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:54.240241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:54.294389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1034/2092] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:54.304189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:54.305747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:54.363940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d181f-17fa-439b-b886-729e286505f0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:54.367615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:54.368216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:54.368496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:54.368771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:54.378705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:54.380249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:54.391105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1059/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:54.470058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:54.470058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:54.480058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dd3f2-ae01-4d56-9c2c-48ad4d7f775b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:54.485360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:54.486133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:54.486480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:54.486814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] PATH_INFO: `/v3/users/4ae07662def846e399f4dea29d71f58c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:54.488037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae07662def846e399f4dea29d71f58c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:54.490083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:54.842700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae07662def846e399f4dea29d71f58c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.843415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae07662def846e399f4dea29d71f58c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.851204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae07662def846e399f4dea29d71f58c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.851930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae07662def846e399f4dea29d71f58c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.985100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a863e46f-684b-4bad-9a4c-467f88c0edce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae07662def846e399f4dea29d71f58c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:54.993460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:54.995318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:55.008713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1035/2094] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:54 2022] DELETE /identity/v3/users/4ae07662def846e399f4dea29d71f58c => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:55.013664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.014179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.014600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.035897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:55.037165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:55.053070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf409-d1d7-4642-b8e6-81baf35817bf service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.078969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:55.109091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1036/2095] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:55.109494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.110562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.111331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.111821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] PATH_INFO: `/v3/projects/9cd88d8702cd493d956ee8d024362b40` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.113090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd88d8702cd493d956ee8d024362b40)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.115083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.119076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.133715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:55.150391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:55.184818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98f131-53a1-4a87-bb37-dd511de8077d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.188519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd7e7-8131-47e9-a1b9-ad1b8c6f1829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.189167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd7e7-8131-47e9-a1b9-ad1b8c6f1829 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.189445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd7e7-8131-47e9-a1b9-ad1b8c6f1829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.189718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd7e7-8131-47e9-a1b9-ad1b8c6f1829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.199347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cd7e7-8131-47e9-a1b9-ad1b8c6f1829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.200772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cd7e7-8131-47e9-a1b9-ad1b8c6f1829 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.210710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1037/2096] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:55.248521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd88d8702cd493d956ee8d024362b40', 'request_id': 'req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:55.254880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd88d8702cd493d956ee8d024362b40', 'request_id': 'req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:55.349972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd88d8702cd493d956ee8d024362b40', 'request_id': 'req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:55.361917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] The token cache is being invalidate because project 9cd88d8702cd493d956ee8d024362b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:55.362219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:55.362997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ccfd1f5a-4200-4bfe-90b7-6604b337f92e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:55.364452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1060/2097] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:54 2022] DELETE /identity/v3/projects/9cd88d8702cd493d956ee8d024362b40 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:55.369469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.369980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.370196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.390864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:55.392082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:55.469201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1038/2098] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:55.477209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.477452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.477735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.499889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:55.501244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:55.509402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1061/2099] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:55.513916 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.523402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d4e40-6a48-4327-a6a5-0e0e2705ee04 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.529272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.529893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.530172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.530446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.531101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:55.531402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.533041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.545630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1039/2100] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:55.550275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.560242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c12d15-de3d-46d8-b65b-f27465859d7f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.564979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.565584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.565864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.566136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.566771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.568706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.688431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.690471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:55.691927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1062/2101] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:55 2022] POST /identity/v3/projects => generated 399 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:55.697665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.712369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309e664-18a9-4a23-8de9-f2c9ef415a0e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.785295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e68d2-ce3f-4b7a-997c-d1fdf1e66cc5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.790277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.790959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.791229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.791335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.791497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.792205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.792326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.792708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.793042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] PATH_INFO: `/v3/projects/4a52bd921146485fb107be1607f9b7ed` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.794155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.794279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a52bd921146485fb107be1607f9b7ed)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.796084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.918719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1063/2102] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:55 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:55.919809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.929808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6683-1279-402a-b394-41c7d19fa577 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.934960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.935562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.935849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.936127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.936817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.939072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.952762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1064/2103] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:36:55.953790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.963968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d081b-b9bc-436d-ba5e-cfb7a063c1df admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:55.969118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:55.970238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:55.970683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:55.970971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:55.971685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:55.973332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:55.979559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:55.986291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1065/2104] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:55.989148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a52bd921146485fb107be1607f9b7ed', 'request_id': 'req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:55.990233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:55.996694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a52bd921146485fb107be1607f9b7ed', 'request_id': 'req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.002645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c57a36-18a4-4a0f-b6b9-2417a3d7a78c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:56.003132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a52bd921146485fb107be1607f9b7ed', 'request_id': 'req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.076033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd5378-4a7d-41dd-9d6f-105d5e4db78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:56.076709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd5378-4a7d-41dd-9d6f-105d5e4db78c admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:56.077020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd5378-4a7d-41dd-9d6f-105d5e4db78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:56.077301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd5378-4a7d-41dd-9d6f-105d5e4db78c admin admin] PATH_INFO: `/v3/projects/da2b6a5858e94e43872afa238977b1a7/users/a92be18cd2e84f01896576c02665aa3d/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:56.097767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd5378-4a7d-41dd-9d6f-105d5e4db78c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da2b6a5858e94e43872afa238977b1a7, user_id=a92be18cd2e84f01896576c02665aa3d, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:56.101610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd5378-4a7d-41dd-9d6f-105d5e4db78c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:56.121371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:56.182333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] The token cache is being invalidate because project 4a52bd921146485fb107be1607f9b7ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:56.182870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.183524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.185118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1040/2105] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:55 2022] DELETE /identity/v3/projects/4a52bd921146485fb107be1607f9b7ed => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:56.189106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:56.190419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:56.203963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da7a9d3-cf46-42ec-a1a9-8492ea4f1e0d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:56.209055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:56.209688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:56.209973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:56.210250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] PATH_INFO: `/v3/users/97b29f169f344bec862f29352b59f236` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:56.211428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b29f169f344bec862f29352b59f236)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:56.213398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:56.216759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1066/2106] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:55 2022] PUT /identity/v3/projects/da2b6a5858e94e43872afa238977b1a7/users/a92be18cd2e84f01896576c02665aa3d/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:56.226877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:56.227165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:56.227447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:56.255728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] MFA Rules not processed for user `a92be18cd2e84f01896576c02665aa3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:56.256903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:56.319345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1067/2107] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:36:56.331292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:56.332644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:56.389763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd66c5-df59-4d66-a281-fed0f7d049c0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:56.393388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:56.394076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:56.394355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:56.394629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:56.402892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:56.404319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:56.413143 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1068/2108] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:56 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:36:56.467991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b29f169f344bec862f29352b59f236'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.468668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b29f169f344bec862f29352b59f236'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.476057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b29f169f344bec862f29352b59f236'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.476712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b29f169f344bec862f29352b59f236'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.589637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c6c64beb-c7f3-4c3b-b56b-6dee2a8c309d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b29f169f344bec862f29352b59f236'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:56.616599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1041/2109] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:56 2022] DELETE /identity/v3/users/97b29f169f344bec862f29352b59f236 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:36:57.117274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:57.143981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:57.202282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc13bb-9d6e-4b84-bdee-38d4ba1e7bda service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:57.271429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:57.272195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:57.272480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:57.272754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] PATH_INFO: `/v3/projects/65cb532e7b7e40e98947d46b7e374e1d` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:57.274189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65cb532e7b7e40e98947d46b7e374e1d)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:57.276672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:57.440352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65cb532e7b7e40e98947d46b7e374e1d', 'request_id': 'req-9b85d2d5-5f25-439f-bf9e-d33623396c42'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:57.446377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65cb532e7b7e40e98947d46b7e374e1d', 'request_id': 'req-9b85d2d5-5f25-439f-bf9e-d33623396c42'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:57.527586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65cb532e7b7e40e98947d46b7e374e1d', 'request_id': 'req-9b85d2d5-5f25-439f-bf9e-d33623396c42'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:57.539476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] The token cache is being invalidate because project 65cb532e7b7e40e98947d46b7e374e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:57.539764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:57.540398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9b85d2d5-5f25-439f-bf9e-d33623396c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:57.542291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1069/2110] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:57 2022] DELETE /identity/v3/projects/65cb532e7b7e40e98947d46b7e374e1d => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:57.546905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:57.547369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:57.547489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:57.570766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:57.572421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:57.646409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1042/2111] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:57.657348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:57.657348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:57.657348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:57.679376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:57.680680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:57.692718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1070/2112] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:57.697149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:57.707509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcc22f-d28d-4036-805b-3e9f3ceb172f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:57.712376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:57.712999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:57.713278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:57.713690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:57.714401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:57.714699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:57.716287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:57.727952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1043/2113] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:57.732980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:57.742374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a55cf-dbc2-4743-bf5f-9eff60c01fe0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:57.749078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:57.749690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:57.750025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:57.750301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:57.750981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:57.753009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:57.872146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1071/2114] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:57 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:57.876458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:57.887730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9579abb-3ff4-4ecb-9330-2970575e7bdc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:57.930687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:57.948090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:57.948844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:57.949163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:57.949441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:57.950087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:57.952020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.040319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1044/2115] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:57 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:58.044356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.063342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497c716-24b5-4549-90a1-5f50b6d8504c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.068110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.068715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.069027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.069305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.069952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.071561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.082181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1072/2116] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:58.086641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.096927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf545b-441c-4373-a427-5be35bbebe97 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.108790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84260891-59de-47f1-8f10-dbb32e6a524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.108790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260891-59de-47f1-8f10-dbb32e6a524d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.108790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260891-59de-47f1-8f10-dbb32e6a524d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.108790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84260891-59de-47f1-8f10-dbb32e6a524d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.109176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84260891-59de-47f1-8f10-dbb32e6a524d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.109176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84260891-59de-47f1-8f10-dbb32e6a524d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.116118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1045/2117] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:58.120668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.130895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d70d2f-4413-488b-965e-cd3de5871098 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.135745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.143162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.143162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.143162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] PATH_INFO: `/v3/projects/c3a9624850944f1ca5fd9a6b2d78bf8a/users/9f62868f9a9c44b4bd4153c762fdda82/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.149098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:58.153817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a9624850944f1ca5fd9a6b2d78bf8a, user_id=9f62868f9a9c44b4bd4153c762fdda82, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.156555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.264284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1073/2118] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:58 2022] PUT /identity/v3/projects/c3a9624850944f1ca5fd9a6b2d78bf8a/users/9f62868f9a9c44b4bd4153c762fdda82/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:58.275076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.275365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.275693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.306004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] MFA Rules not processed for user `9f62868f9a9c44b4bd4153c762fdda82`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:58.307361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:58.381063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1046/2119] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:58.385523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.455934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad668d8-9597-4500-8d10-f9ac5f0924ce admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.461975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.462760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.463033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.464917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.465661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.468744 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.567671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1074/2120] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:58 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:58.571882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.583463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb96db-e6d9-4c8f-9075-6c62a6fb01a8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.654626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.655166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.655453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.655728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.656413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.658471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.808122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1047/2121] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:58 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:58.811513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.821845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1adbb5-6671-4a48-b8f0-688265991dc6 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.827122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.827720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.828029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.828303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.828979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.830589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.841903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1075/2122] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:58.845919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.856829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0763c-e5eb-4ff7-94a3-6f8085ba3d0d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.861758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a47ab-778a-4eda-aa2d-f4f93c1df2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.862408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a47ab-778a-4eda-aa2d-f4f93c1df2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.862686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a47ab-778a-4eda-aa2d-f4f93c1df2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.863004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a47ab-778a-4eda-aa2d-f4f93c1df2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.863668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a47ab-778a-4eda-aa2d-f4f93c1df2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.865312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a47ab-778a-4eda-aa2d-f4f93c1df2ba admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:58.876230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1048/2123] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:58.880367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:58.891938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972576-b73f-403d-bc2c-87da14248de0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:58.896983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:58.897697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:58.897979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:58.898256 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] PATH_INFO: `/v3/projects/9586fe2bbbc44ae1ba4bb957e81e7dda/users/4af8f738ed2e4d558e4b94130047f977/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:58.919138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9586fe2bbbc44ae1ba4bb957e81e7dda, user_id=4af8f738ed2e4d558e4b94130047f977, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:58.922051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:59.052806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1076/2124] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:58 2022] PUT /identity/v3/projects/9586fe2bbbc44ae1ba4bb957e81e7dda/users/4af8f738ed2e4d558e4b94130047f977/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:59.063670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:59.063956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:59.064234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:59.095408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] MFA Rules not processed for user `4af8f738ed2e4d558e4b94130047f977`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:59.096627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:59.115815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:59.162063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1049/2125] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:59.167261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:59.237167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8819e5-6d54-4641-bff0-1a1184eab571 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:59.242240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:59.242848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:59.243130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:59.243409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:59.244057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:59.246145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:59.545724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1077/2126] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:59 2022] POST /identity/v3/projects => generated 377 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:59.549838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:59.561063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785eb598-a605-4a30-9cdd-b078fdba2a3d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:59.631265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:59.632030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:59.632339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:59.632612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:59.633292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:59.635987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:59.819333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1050/2127] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:59 2022] POST /identity/v3/users => generated 295 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:59.823331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:59.837001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb72843-8d87-43ba-90be-cd273f38539c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:59.843518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:59.844138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:59.844430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:59.844707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:59.845489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:59.847978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:59.861086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1078/2128] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:59.865525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:59.876829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edaf2b-916d-4933-b7c5-385958a6323e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:59.882358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:59.883030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:59.883308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:59.883590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] PATH_INFO: `/v3/projects/e04f9e97da5441fd992d98e49d9d8335/users/aaf9d7c6345b4ae4ad5158f1a944651b/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:59.902767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04f9e97da5441fd992d98e49d9d8335, user_id=aaf9d7c6345b4ae4ad5158f1a944651b, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:59.907634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:00.030685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1051/2129] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:59 2022] PUT /identity/v3/projects/e04f9e97da5441fd992d98e49d9d8335/users/aaf9d7c6345b4ae4ad5158f1a944651b/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:00.038474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:00.058219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead34d8a-36cc-40f4-9612-08f7f207aeb9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:00.070548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:00.133595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c287c3d-a745-43b9-b147-68118a313027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:00.134373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c287c3d-a745-43b9-b147-68118a313027 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:00.134895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c287c3d-a745-43b9-b147-68118a313027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:00.135865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c287c3d-a745-43b9-b147-68118a313027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:00.137160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c287c3d-a745-43b9-b147-68118a313027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:00.142610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c287c3d-a745-43b9-b147-68118a313027 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:00.157159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1079/2130] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:37:00.170094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:00.183669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef887bb9-5f22-4eb9-b7af-8d2398c532b3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:00.192119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:00.193025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:00.193797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:00.194869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] PATH_INFO: `/v3/projects/e04f9e97da5441fd992d98e49d9d8335/users/aaf9d7c6345b4ae4ad5158f1a944651b/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:00.199396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04f9e97da5441fd992d98e49d9d8335, user_id=aaf9d7c6345b4ae4ad5158f1a944651b, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:00.203820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:00.218445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1052/2131] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:37:00 2022] PUT /identity/v3/projects/e04f9e97da5441fd992d98e49d9d8335/users/aaf9d7c6345b4ae4ad5158f1a944651b/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:00.300577 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:00.301169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:00.301262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:00.337897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] MFA Rules not processed for user `aaf9d7c6345b4ae4ad5158f1a944651b`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:00.338366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:00.423352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1080/2132] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:00.436437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:00.442276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:00.521949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef684528-70d5-4110-b08f-3681b08fb8d8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:00.527690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:00.528325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:00.528605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:00.528913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:00.563875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:00.613881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:00.615326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:00.629899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1053/2133] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:37:03.989677 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:03.996903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a374de-ddde-4958-a994-f3c4aa82d135 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:04.000934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8797ef6-06a2-427e-8889-4c80e99639bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:04.001542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797ef6-06a2-427e-8889-4c80e99639bc service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:04.001890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797ef6-06a2-427e-8889-4c80e99639bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:04.002176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797ef6-06a2-427e-8889-4c80e99639bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:04.065271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8797ef6-06a2-427e-8889-4c80e99639bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:04.066920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8797ef6-06a2-427e-8889-4c80e99639bc service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:04.078556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1081/2134] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:06.206070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:06.207780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:06.289515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060112d-fc91-49da-8269-3955d485eeda service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:06.295548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:06.296306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:06.296691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:06.297117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] PATH_INFO: `/v3/users/a92be18cd2e84f01896576c02665aa3d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:06.298375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92be18cd2e84f01896576c02665aa3d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:06.300432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:06.352042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:06.352519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:06.352708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:06.372785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] MFA Rules not processed for user `961ed374bc664b299c1a6425dc803dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:06.374023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:06.408055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:06.440598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1082/2135] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:37:06.446269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:06.459113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b0713-a4ef-4652-9ea7-ffc22c75aad9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:06.463119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd256ba-f616-422c-92db-9ed58fc7be89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:06.464628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd256ba-f616-422c-92db-9ed58fc7be89 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:06.464628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd256ba-f616-422c-92db-9ed58fc7be89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:06.464628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd256ba-f616-422c-92db-9ed58fc7be89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:06.474758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd256ba-f616-422c-92db-9ed58fc7be89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:06.476300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd256ba-f616-422c-92db-9ed58fc7be89 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:06.494883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1083/2136] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:06.579952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92be18cd2e84f01896576c02665aa3d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:06.580786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92be18cd2e84f01896576c02665aa3d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:06.589304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92be18cd2e84f01896576c02665aa3d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:06.590070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92be18cd2e84f01896576c02665aa3d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:06.703751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92be18cd2e84f01896576c02665aa3d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:06.728451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1054/2137] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:06 2022] DELETE /identity/v3/users/a92be18cd2e84f01896576c02665aa3d => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:06.733972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:06.734701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:06.736181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:06.762441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:06.767754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:06.862658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1084/2138] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:06.871939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:06.874028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:06.931041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:06.937021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc64524-82a3-4347-b1f5-93975ac834e2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:06.940802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:06.941446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:06.941726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:06.941998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:06.951358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:06.952841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:06.963558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1055/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:07.770003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:07.772106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:07.785893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083d03c-d17e-478b-8a54-595a54cf83c0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:07.798818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:07.799483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:07.799926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:07.800204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] PATH_INFO: `/v3/projects/da2b6a5858e94e43872afa238977b1a7` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:07.801413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da2b6a5858e94e43872afa238977b1a7)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:07.803751 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:07.994920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da2b6a5858e94e43872afa238977b1a7', 'request_id': 'req-5519545c-308b-48c7-964f-e2211b24f402'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:08.003326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da2b6a5858e94e43872afa238977b1a7', 'request_id': 'req-5519545c-308b-48c7-964f-e2211b24f402'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:08.098160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da2b6a5858e94e43872afa238977b1a7', 'request_id': 'req-5519545c-308b-48c7-964f-e2211b24f402'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:08.109842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] The token cache is being invalidate because project da2b6a5858e94e43872afa238977b1a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:08.110154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:08.110794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5519545c-308b-48c7-964f-e2211b24f402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:08.112284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1085/2140] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:07 2022] DELETE /identity/v3/projects/da2b6a5858e94e43872afa238977b1a7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:11.680987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:11.684031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:11.701082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2aeb-7708-4c2d-907f-908e4f67f340 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:11.786359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:11.787067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:11.787350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:11.787718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] PATH_INFO: `/v3/users/9f62868f9a9c44b4bd4153c762fdda82` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:11.788800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f62868f9a9c44b4bd4153c762fdda82)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:11.790779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:12.194568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f62868f9a9c44b4bd4153c762fdda82'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:12.195544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f62868f9a9c44b4bd4153c762fdda82'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:12.209287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f62868f9a9c44b4bd4153c762fdda82'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:12.210028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f62868f9a9c44b4bd4153c762fdda82'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:12.349710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f62868f9a9c44b4bd4153c762fdda82'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:12.378731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1056/2141] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:11 2022] DELETE /identity/v3/users/9f62868f9a9c44b4bd4153c762fdda82 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:12.383183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:12.383716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:12.384708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:12.406067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:12.407379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:12.457701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:12.475064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1086/2142] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:12.485816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:12.486451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:12.583531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8a630-5181-457b-8fe6-dae6d143a4cc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:12.583531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:12.583531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:12.583531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:12.584160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:12.584160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:12.584160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:12.590073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1057/2143] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:13.145625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:13.147080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:13.161943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a583bd-4e30-4d8d-9436-ea452a1bd235 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:13.164215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:13.164848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:13.165151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:13.165424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] PATH_INFO: `/v3/projects/c3a9624850944f1ca5fd9a6b2d78bf8a` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:13.167414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a9624850944f1ca5fd9a6b2d78bf8a)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:13.169370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:13.293675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a9624850944f1ca5fd9a6b2d78bf8a', 'request_id': 'req-bb63c48e-eb32-4a86-8732-00e79ff998db'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.300077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a9624850944f1ca5fd9a6b2d78bf8a', 'request_id': 'req-bb63c48e-eb32-4a86-8732-00e79ff998db'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.411246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a9624850944f1ca5fd9a6b2d78bf8a', 'request_id': 'req-bb63c48e-eb32-4a86-8732-00e79ff998db'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.422479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] The token cache is being invalidate because project c3a9624850944f1ca5fd9a6b2d78bf8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:13.422764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.423380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.424840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1087/2144] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:13 2022] DELETE /identity/v3/projects/c3a9624850944f1ca5fd9a6b2d78bf8a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:13.429748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:13.439650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac868d6-77eb-4f7a-8862-5b831c0d6fba service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:13.447285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:13.497298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:13.497927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:13.498209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:13.498482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] PATH_INFO: `/v3/users/4af8f738ed2e4d558e4b94130047f977` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:13.499642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4af8f738ed2e4d558e4b94130047f977)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:13.501627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:13.815594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4af8f738ed2e4d558e4b94130047f977'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.816255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4af8f738ed2e4d558e4b94130047f977'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.824976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4af8f738ed2e4d558e4b94130047f977'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.827095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4af8f738ed2e4d558e4b94130047f977'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.926206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4af8f738ed2e4d558e4b94130047f977'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:13.945176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1058/2145] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:13 2022] DELETE /identity/v3/users/4af8f738ed2e4d558e4b94130047f977 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:14.380670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:14.383037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:14.408177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63c48e-eb32-4a86-8732-00e79ff998db admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:14.474573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:14.475286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:14.475540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:14.475797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] PATH_INFO: `/v3/projects/9586fe2bbbc44ae1ba4bb957e81e7dda` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:14.476962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9586fe2bbbc44ae1ba4bb957e81e7dda)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:14.478974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:14.601792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9586fe2bbbc44ae1ba4bb957e81e7dda', 'request_id': 'req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:14.608086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9586fe2bbbc44ae1ba4bb957e81e7dda', 'request_id': 'req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:14.703739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9586fe2bbbc44ae1ba4bb957e81e7dda', 'request_id': 'req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:14.715401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] The token cache is being invalidate because project 9586fe2bbbc44ae1ba4bb957e81e7dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:14.715833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:14.716590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:14.718262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1088/2146] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:14 2022] DELETE /identity/v3/projects/9586fe2bbbc44ae1ba4bb957e81e7dda => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:14.722404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:14.724002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:14.737180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad879224-cf5b-4e8d-a281-b564881bb400 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:14.799283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:14.799922 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:14.800201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:14.800472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] PATH_INFO: `/v3/users/aaf9d7c6345b4ae4ad5158f1a944651b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:14.801654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf9d7c6345b4ae4ad5158f1a944651b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:14.803698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:14.985344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:14.997928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf9d7c6345b4ae4ad5158f1a944651b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:14.998599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf9d7c6345b4ae4ad5158f1a944651b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.005721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf9d7c6345b4ae4ad5158f1a944651b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.006386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf9d7c6345b4ae4ad5158f1a944651b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.069384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e0f3e7df-deaf-47e2-9a3b-a7cdf0f93992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf9d7c6345b4ae4ad5158f1a944651b'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.088217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1059/2147] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:14 2022] DELETE /identity/v3/users/aaf9d7c6345b4ae4ad5158f1a944651b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:15.611696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:15.611696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:15.620733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:15.630959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580f13a-8b6d-491c-8a6d-c0e6b868b0f2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:15.726501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:15.727469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:15.728432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:15.729771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] PATH_INFO: `/v3/projects/e04f9e97da5441fd992d98e49d9d8335` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:15.731939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04f9e97da5441fd992d98e49d9d8335)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:15.734868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:15.882137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04f9e97da5441fd992d98e49d9d8335', 'request_id': 'req-7661eee6-c507-49a8-bfb4-82c4af739fbb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.890470 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04f9e97da5441fd992d98e49d9d8335', 'request_id': 'req-7661eee6-c507-49a8-bfb4-82c4af739fbb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.968655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04f9e97da5441fd992d98e49d9d8335', 'request_id': 'req-7661eee6-c507-49a8-bfb4-82c4af739fbb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.998612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] The token cache is being invalidate because project e04f9e97da5441fd992d98e49d9d8335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:15.998993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:15.999878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7661eee6-c507-49a8-bfb4-82c4af739fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:16.001530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1089/2148] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:15 2022] DELETE /identity/v3/projects/e04f9e97da5441fd992d98e49d9d8335 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:16.006762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.006762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.007331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.035677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:16.037258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:16.100167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1060/2149] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:37:16.108061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.108293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.108558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.131688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:16.132958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:16.144180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1090/2150] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:16.148754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.168732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144766b6-a44e-4453-8247-2d4d45996ee8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.174436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.175057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.175337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.175611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.176531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:37:16.176845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.179234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.192753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1061/2151] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:37:16.198798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.207830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1a3b6-6258-47c6-8522-3b94650ad9b8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.215131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.215527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.215788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.216058 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.216688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.218611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.345714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1091/2152] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:37:16 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:37:16.349948 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.361656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb90207-2c01-44d2-9cee-e28253632569 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.419985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.420641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.420949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.421225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.422048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.423959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.521203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1062/2153] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:37:16 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:37:16.525702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.538804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48f195-73a1-4ebc-b541-e4a6ea13a9e4 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.545411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.546075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.546371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.546683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.547324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.549075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.560898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1092/2154] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:37:16.570706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.581450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e5552-b807-4f12-b4d1-fc2deab76047 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.586359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddb2d0-58dc-4c57-9139-2faac41719ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.586956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb2d0-58dc-4c57-9139-2faac41719ee admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.587441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb2d0-58dc-4c57-9139-2faac41719ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.587551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddb2d0-58dc-4c57-9139-2faac41719ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.588630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddb2d0-58dc-4c57-9139-2faac41719ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.590276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddb2d0-58dc-4c57-9139-2faac41719ee admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.600911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1063/2155] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:37:16.605680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.616343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79928a4b-0064-477b-935a-62360bc5fa40 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.622291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.622956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.625304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.625304 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] PATH_INFO: `/v3/projects/2226c5da9f2b46a89b38074a02442984/users/443b8915508f4f22bcce40dc0ad0cd5e/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.639036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2226c5da9f2b46a89b38074a02442984, user_id=443b8915508f4f22bcce40dc0ad0cd5e, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.641738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.765830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1093/2156] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:37:16 2022] PUT /identity/v3/projects/2226c5da9f2b46a89b38074a02442984/users/443b8915508f4f22bcce40dc0ad0cd5e/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:16.777749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.778035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.778312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.808054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] MFA Rules not processed for user `443b8915508f4f22bcce40dc0ad0cd5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:16.809306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:16.813307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:16.876687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1064/2157] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:16.888087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:16.889977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:16.923861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:16.950374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d306005-9f03-4666-849c-9688b7a07c0e admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:16.954501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de46026-a11f-4c0e-918d-b0015926ce6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:16.955266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de46026-a11f-4c0e-918d-b0015926ce6c service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:16.955892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de46026-a11f-4c0e-918d-b0015926ce6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:16.956289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de46026-a11f-4c0e-918d-b0015926ce6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:16.965769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de46026-a11f-4c0e-918d-b0015926ce6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:16.968565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de46026-a11f-4c0e-918d-b0015926ce6c service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:16.975748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1094/2158] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:17.001674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:17.072674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ab47e-6998-467f-a2a6-eb1542944426 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:17.077540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:17.078180 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:17.078464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:17.078746 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] PATH_INFO: `/v3/users/443b8915508f4f22bcce40dc0ad0cd5e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:17.079899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=443b8915508f4f22bcce40dc0ad0cd5e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:17.081887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:17.337096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443b8915508f4f22bcce40dc0ad0cd5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:17.337745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443b8915508f4f22bcce40dc0ad0cd5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:17.346109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443b8915508f4f22bcce40dc0ad0cd5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:17.346936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443b8915508f4f22bcce40dc0ad0cd5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:17.452182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443b8915508f4f22bcce40dc0ad0cd5e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:17.491063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1065/2159] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:17 2022] DELETE /identity/v3/users/443b8915508f4f22bcce40dc0ad0cd5e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:17.496079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:17.496362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:17.496692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:17.516054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:17.517542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:17.577074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1095/2160] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:17.586992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:17.592236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:17.619323 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:17.649973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8297516-1167-47bc-bb0a-eceed21cab2d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:17.653846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b04b62-98bc-469b-80e1-98a0a6476236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:17.654393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b04b62-98bc-469b-80e1-98a0a6476236 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:17.654667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b04b62-98bc-469b-80e1-98a0a6476236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:17.654999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b04b62-98bc-469b-80e1-98a0a6476236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:17.663964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b04b62-98bc-469b-80e1-98a0a6476236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:17.665410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b04b62-98bc-469b-80e1-98a0a6476236 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:17.674972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1066/2161] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:18.076802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:18.078571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:18.093010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1156e7-e06c-4876-b1fc-da249352a135 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:18.097796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:18.098555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:18.099338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:18.099338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] PATH_INFO: `/v3/projects/2226c5da9f2b46a89b38074a02442984` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:18.100533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2226c5da9f2b46a89b38074a02442984)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:18.102617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:18.213812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2226c5da9f2b46a89b38074a02442984', 'request_id': 'req-07536957-bc4f-4fd0-a1b4-1b90846c85a3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:18.220964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2226c5da9f2b46a89b38074a02442984', 'request_id': 'req-07536957-bc4f-4fd0-a1b4-1b90846c85a3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:18.355280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2226c5da9f2b46a89b38074a02442984', 'request_id': 'req-07536957-bc4f-4fd0-a1b4-1b90846c85a3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:18.361926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] The token cache is being invalidate because project 2226c5da9f2b46a89b38074a02442984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:18.362210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:18.362872 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-07536957-bc4f-4fd0-a1b4-1b90846c85a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:18.369146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1096/2162] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:18 2022] DELETE /identity/v3/projects/2226c5da9f2b46a89b38074a02442984 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:26.763877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:26.764390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:26.764823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:26.799735 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] MFA Rules not processed for user `c4f4844eb5e14d789476cfb44e2516f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:26.800988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:26.864539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1067/2163] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:28.358878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:28.478323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:28.478658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:28.478931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:28.508619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:28.508619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:28.575864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1097/2164] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:37:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:28.621446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:28.626742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:28.724481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f22d9-b370-4c1e-9171-f6a82c4a70b4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:28.730733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353b5d2-3cf6-4d2a-b12e-8b7b8cccfc0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:28.731456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353b5d2-3cf6-4d2a-b12e-8b7b8cccfc0e service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:28.731796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353b5d2-3cf6-4d2a-b12e-8b7b8cccfc0e service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:28.732128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353b5d2-3cf6-4d2a-b12e-8b7b8cccfc0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:28.741074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353b5d2-3cf6-4d2a-b12e-8b7b8cccfc0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:28.742992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353b5d2-3cf6-4d2a-b12e-8b7b8cccfc0e service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:28.752157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1068/2165] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:37:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:37:38.748834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=72 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:39.234954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=25 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:40.427980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:40.429600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:40.509698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8de62f-713d-4e00-bae5-7c8e2d54805d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:40.517784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:40.517784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:40.517784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:40.517784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] PATH_INFO: `/v3/users/5d09dd5e28af4dea9a744247449311ae` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:40.519579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d09dd5e28af4dea9a744247449311ae)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:40.519579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:40.772676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d09dd5e28af4dea9a744247449311ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:40.773417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d09dd5e28af4dea9a744247449311ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:40.784900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d09dd5e28af4dea9a744247449311ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:40.784900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d09dd5e28af4dea9a744247449311ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:40.892550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d09dd5e28af4dea9a744247449311ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:40.926988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1098/2166] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:40 2022] DELETE /identity/v3/users/5d09dd5e28af4dea9a744247449311ae => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:37:40.932353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:40.932888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:40.933003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:40.961267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:40.963157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:41.094551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1069/2167] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:41.133984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:41.136279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:41.252668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b2d33-ed6b-45d5-a815-5b15c1e7f06c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:41.256445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:41.257098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:41.257380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:41.257654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:41.268871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:41.270348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:41.283228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1099/2168] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:41.967818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:41.969345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:41.982757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87e75d-7ed8-4737-b549-794b3b0931b1 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:41.988096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:41.988775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:41.989106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:41.989384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] PATH_INFO: `/v3/projects/a027b44d2783413ba8ed033b81db4920` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:41.991312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a027b44d2783413ba8ed033b81db4920)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:41.993865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:42.103485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a027b44d2783413ba8ed033b81db4920', 'request_id': 'req-2ab2b332-127e-4d69-b352-c307b8b7c632'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.110615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a027b44d2783413ba8ed033b81db4920', 'request_id': 'req-2ab2b332-127e-4d69-b352-c307b8b7c632'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.116346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a027b44d2783413ba8ed033b81db4920', 'request_id': 'req-2ab2b332-127e-4d69-b352-c307b8b7c632'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.188994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] The token cache is being invalidate because project a027b44d2783413ba8ed033b81db4920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:42.189548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.190227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.191754 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1070/2169] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:41 2022] DELETE /identity/v3/projects/a027b44d2783413ba8ed033b81db4920 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:37:42.196670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:42.209777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481c2a5-ac16-44bd-a51a-592d4b8534d1 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:42.229298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:42.278871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:42.280042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:42.280735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:42.280985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] PATH_INFO: `/v3/users/6d3e9f82720e4f2d93620b79f8e79ef1` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:42.282539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3e9f82720e4f2d93620b79f8e79ef1)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:42.284519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:42.529543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3e9f82720e4f2d93620b79f8e79ef1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.530241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3e9f82720e4f2d93620b79f8e79ef1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.537358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3e9f82720e4f2d93620b79f8e79ef1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.538014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3e9f82720e4f2d93620b79f8e79ef1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.674758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-fdf3c2ff-3a62-47e9-a78c-813cde3f0e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3e9f82720e4f2d93620b79f8e79ef1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:42.693262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1100/2170] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:42 2022] DELETE /identity/v3/users/6d3e9f82720e4f2d93620b79f8e79ef1 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:43.275221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:43.275221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:43.291317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab2b332-127e-4d69-b352-c307b8b7c632 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:43.361434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:43.362292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:43.362640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:43.362926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] PATH_INFO: `/v3/projects/fddaaa99777d42f4b77ec1ade3fc3c81` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:43.364070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fddaaa99777d42f4b77ec1ade3fc3c81)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:43.366396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:43.448351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:43.474529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fddaaa99777d42f4b77ec1ade3fc3c81', 'request_id': 'req-90556afc-8f6e-4dc4-8e21-2f2ff459540c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:43.481385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fddaaa99777d42f4b77ec1ade3fc3c81', 'request_id': 'req-90556afc-8f6e-4dc4-8e21-2f2ff459540c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:43.487622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fddaaa99777d42f4b77ec1ade3fc3c81', 'request_id': 'req-90556afc-8f6e-4dc4-8e21-2f2ff459540c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:43.566442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] The token cache is being invalidate because project fddaaa99777d42f4b77ec1ade3fc3c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:43.566741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:43.567376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-90556afc-8f6e-4dc4-8e21-2f2ff459540c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:43.568853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1071/2171] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:43 2022] DELETE /identity/v3/projects/fddaaa99777d42f4b77ec1ade3fc3c81 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:43.573666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:43.574782 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:43.574928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:43.596305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:43.597564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:43.659386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1101/2172] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:43.668721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:43.669061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:43.669341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:43.689832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:43.691052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:43.698993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1072/2173] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:43.703798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:43.713711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35e5f6-d474-4587-8e5e-ac045a7fb33b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:43.719875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:43.720582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:43.721118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:43.721516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:43.722162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:37:43.722464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:43.725559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:43.737854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1102/2174] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:37:43.742086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:43.752071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c9616-8829-4504-b321-8e9cc5bd4565 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:43.757052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:43.757652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:43.757933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:43.758207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:43.758875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:43.760788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:43.911295 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1073/2175] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:37:43 2022] POST /identity/v3/projects => generated 409 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:37:43.915440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:43.925581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80923fba-54ea-4eea-b1d7-71b9a1cd7fca admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:43.952065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:43.982026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:43.982631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:43.982918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:43.983192 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:43.983828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:43.985831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.085926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1103/2176] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:37:43 2022] POST /identity/v3/users => generated 305 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:37:44.090245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.101944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fdd98-06ff-43c3-abc6-bafa7ea50baf admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.106951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.107552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.107851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.108107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.108745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.110366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.120662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1074/2177] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:37:44.126084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.136318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768b302-d64c-4f11-b63c-6270a9f55b4a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.141780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdcb8f-b035-48bd-b0fa-adb2684d8daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.142382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcb8f-b035-48bd-b0fa-adb2684d8daf admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.142687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcb8f-b035-48bd-b0fa-adb2684d8daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.142967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdcb8f-b035-48bd-b0fa-adb2684d8daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.143617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdcb8f-b035-48bd-b0fa-adb2684d8daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.145301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdcb8f-b035-48bd-b0fa-adb2684d8daf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.155646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1104/2178] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:37:44.159845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.170757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f9c1-b689-4d92-a9a0-8ca8cadfac82 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.175552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.176210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.176495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.176774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] PATH_INFO: `/v3/projects/b4a187a9753b40f0927027adea8cebac/users/898b17bfde0d4034b3988cbce9b3d209/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.193427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a187a9753b40f0927027adea8cebac, user_id=898b17bfde0d4034b3988cbce9b3d209, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.196100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.315494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1075/2179] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:37:44 2022] PUT /identity/v3/projects/b4a187a9753b40f0927027adea8cebac/users/898b17bfde0d4034b3988cbce9b3d209/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:44.327223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.327514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.327795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.357690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] MFA Rules not processed for user `898b17bfde0d4034b3988cbce9b3d209`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:44.358905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:44.422651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1105/2180] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:44 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:44.427083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.487642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02f98-c2d1-4cb0-83d7-9d1f5de84730 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.492245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.492836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.493140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.493410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.494027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.495911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.623515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1076/2181] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:37:44 2022] POST /identity/v3/projects => generated 409 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:37:44.627778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.640363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926c646-c2ca-47a1-8f98-02a07339010c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.707730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.708659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.709416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.709882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.710555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.712508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.801649 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1106/2182] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:37:44 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:37:44.805602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.816536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c91def-e18c-49a5-90fb-6c869b049e87 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.821391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.825665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.825665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.825665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.825665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.825665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.836232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1077/2183] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:37:44.841764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.852948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c746f8-1b15-4509-b4f6-fad881e0564b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.858355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:44.859171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:44.859452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:44.859729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] PATH_INFO: `/v3/projects/f59794bc82634507a4e89621989b34d9/users/5dcd72fe876f40bc9fb4526e5d5c5c3e/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:44.874490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59794bc82634507a4e89621989b34d9, user_id=5dcd72fe876f40bc9fb4526e5d5c5c3e, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:44.877170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:44.947999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1107/2184] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:37:44 2022] PUT /identity/v3/projects/f59794bc82634507a4e89621989b34d9/users/5dcd72fe876f40bc9fb4526e5d5c5c3e/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:44.952708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:44.964111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782c419-5601-4d23-9545-9e62e2b11061 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:44.985441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:45.023119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efd6d7-fded-4fd2-9c61-cf94e13c6a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:45.023725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efd6d7-fded-4fd2-9c61-cf94e13c6a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:45.024048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efd6d7-fded-4fd2-9c61-cf94e13c6a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:45.024322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efd6d7-fded-4fd2-9c61-cf94e13c6a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:45.024999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efd6d7-fded-4fd2-9c61-cf94e13c6a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:45.026599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efd6d7-fded-4fd2-9c61-cf94e13c6a9b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:45.037288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1078/2185] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:37:45.041591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:45.047432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:45.052614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b645ca-0cab-4d9b-961e-538c77ae598b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:45.057366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:45.058013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:45.058293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:45.058569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] PATH_INFO: `/v3/projects/f59794bc82634507a4e89621989b34d9/users/5dcd72fe876f40bc9fb4526e5d5c5c3e/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:45.060644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59794bc82634507a4e89621989b34d9, user_id=5dcd72fe876f40bc9fb4526e5d5c5c3e, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:45.063335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:45.073388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1108/2186] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:37:45 2022] PUT /identity/v3/projects/f59794bc82634507a4e89621989b34d9/users/5dcd72fe876f40bc9fb4526e5d5c5c3e/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:45.084668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:45.084988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:45.085271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:45.113949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] MFA Rules not processed for user `5dcd72fe876f40bc9fb4526e5d5c5c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:45.117931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:45.184031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1079/2187] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:45 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:45.194726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:45.196546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:45.256091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0175c-b5ca-4414-8be6-66df5857e6ac admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:45.259827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478f801-d8b9-4dc3-9788-bbed8f1a7d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:45.260421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478f801-d8b9-4dc3-9788-bbed8f1a7d05 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:45.260696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478f801-d8b9-4dc3-9788-bbed8f1a7d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:45.261092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478f801-d8b9-4dc3-9788-bbed8f1a7d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:45.270825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478f801-d8b9-4dc3-9788-bbed8f1a7d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:45.272257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478f801-d8b9-4dc3-9788-bbed8f1a7d05 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:45.282930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1109/2188] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:45.600845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:45.611366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95723fb-fa60-4b67-85aa-dae02281fa76 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:45.614945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49d018-e077-4867-ac23-60bb8dca6a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:45.615549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49d018-e077-4867-ac23-60bb8dca6a4d service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:45.615823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49d018-e077-4867-ac23-60bb8dca6a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:45.616093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49d018-e077-4867-ac23-60bb8dca6a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:45.675229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49d018-e077-4867-ac23-60bb8dca6a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:45.676672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49d018-e077-4867-ac23-60bb8dca6a4d service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:45.686395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1080/2189] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:48.226317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433252-fb1a-43e7-9ed3-dad7b1392afe None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.226990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433252-fb1a-43e7-9ed3-dad7b1392afe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.226990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433252-fb1a-43e7-9ed3-dad7b1392afe None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.228327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1110/2190] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:37:48.234353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892c5e-eae7-4738-9676-3d6606a60f7f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.234815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892c5e-eae7-4738-9676-3d6606a60f7f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.235086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c892c5e-eae7-4738-9676-3d6606a60f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.255238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2c892c5e-eae7-4738-9676-3d6606a60f7f None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:48.256454 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2c892c5e-eae7-4738-9676-3d6606a60f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:48.301039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:48.338602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1081/2191] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:48.348689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433252-fb1a-43e7-9ed3-dad7b1392afe None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:48.350862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-87433252-fb1a-43e7-9ed3-dad7b1392afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:48.466527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433252-fb1a-43e7-9ed3-dad7b1392afe None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:48.471128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7c251-e717-4432-8d7c-1feafa6af459 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:48.472031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c251-e717-4432-8d7c-1feafa6af459 service glance] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.472483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c251-e717-4432-8d7c-1feafa6af459 service glance] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.472947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7c251-e717-4432-8d7c-1feafa6af459 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.486912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c251-e717-4432-8d7c-1feafa6af459 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:48.488381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7c251-e717-4432-8d7c-1feafa6af459 service glance] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:48.500890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1111/2192] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:37:48.688085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d1f43-0af0-4eb8-947c-11dfa96ffcac None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.688287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d1f43-0af0-4eb8-947c-11dfa96ffcac None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.688565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d1f43-0af0-4eb8-947c-11dfa96ffcac None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.689969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1082/2193] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:37:48.694719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93bfc-3008-4e5f-bf92-6b9b19201949 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.695095 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93bfc-3008-4e5f-bf92-6b9b19201949 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.695445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93bfc-3008-4e5f-bf92-6b9b19201949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.717069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-57f93bfc-3008-4e5f-bf92-6b9b19201949 None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:48.718277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-57f93bfc-3008-4e5f-bf92-6b9b19201949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:48.771715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1112/2194] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:48.783852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d1f43-0af0-4eb8-947c-11dfa96ffcac None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:48.785675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7e3d1f43-0af0-4eb8-947c-11dfa96ffcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:48.800327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d1f43-0af0-4eb8-947c-11dfa96ffcac None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:48.805562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200047e-3968-45ba-88fd-098e02475e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:48.806306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200047e-3968-45ba-88fd-098e02475e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.806644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200047e-3968-45ba-88fd-098e02475e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.807122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200047e-3968-45ba-88fd-098e02475e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.817302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200047e-3968-45ba-88fd-098e02475e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:48.818844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200047e-3968-45ba-88fd-098e02475e99 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:48.827168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1083/2195] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:48.912723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b673d1-724a-471a-8b37-4cf9ec08167c None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.913303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b673d1-724a-471a-8b37-4cf9ec08167c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.913582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b673d1-724a-471a-8b37-4cf9ec08167c None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.915021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1113/2196] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:37:48.921081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520540e6-ade0-4f73-a10c-b1cd6daf8928 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.921374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520540e6-ade0-4f73-a10c-b1cd6daf8928 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.921655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520540e6-ade0-4f73-a10c-b1cd6daf8928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.942215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-520540e6-ade0-4f73-a10c-b1cd6daf8928 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:48.943433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-520540e6-ade0-4f73-a10c-b1cd6daf8928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:48.952704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1084/2197] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:37:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:48.963624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b673d1-724a-471a-8b37-4cf9ec08167c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:48.974399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b673d1-724a-471a-8b37-4cf9ec08167c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:48.977978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:48.978573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:48.978847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:48.979117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:48.989934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:48.991369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:49.002819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1114/2198] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:55.154837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:55.154837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:55.155401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:55.174475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:55.175628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:55.229789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1085/2199] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:37:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:55.242488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:55.243906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:55.257422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:55.313733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bd1a2-f1a9-413a-bdf2-576ad2b9a47c service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:55.319487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410f39c-8db3-4905-8151-819d7c2f40a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:55.320679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f39c-8db3-4905-8151-819d7c2f40a7 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:55.320954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f39c-8db3-4905-8151-819d7c2f40a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:55.321226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410f39c-8db3-4905-8151-819d7c2f40a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:55.330015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410f39c-8db3-4905-8151-819d7c2f40a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:55.331780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410f39c-8db3-4905-8151-819d7c2f40a7 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:55.340208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1115/2200] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:37:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:37:56.056005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:56.130524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf54ab-82e9-4b96-86ba-fc676350f7cb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:56.137444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:56.138288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:56.138994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:56.139262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] PATH_INFO: `/v3/users/898b17bfde0d4034b3988cbce9b3d209` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:56.140440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=898b17bfde0d4034b3988cbce9b3d209)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:56.142420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:56.351359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '898b17bfde0d4034b3988cbce9b3d209'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:56.352063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '898b17bfde0d4034b3988cbce9b3d209'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:56.360375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '898b17bfde0d4034b3988cbce9b3d209'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:56.361088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '898b17bfde0d4034b3988cbce9b3d209'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:56.423024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '898b17bfde0d4034b3988cbce9b3d209'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:56.424713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:56.447834 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1086/2201] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:56 2022] DELETE /identity/v3/users/898b17bfde0d4034b3988cbce9b3d209 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:56.449525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:56.450142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:56.450385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:56.473560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:56.475208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:56.541476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1116/2202] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:56.551658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:56.553195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:56.614671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f362e-6da2-4aab-9be3-fc7fd002f64b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:56.618568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:56.619510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:56.619887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:56.620223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:56.632574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:56.634164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:56.645875 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1087/2203] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:37:57.249940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:57.249940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:57.276169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8d9ac-a960-4cca-add4-f9dc0ed19c71 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:57.284326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:57.285089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:57.285446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:57.285727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] PATH_INFO: `/v3/projects/b4a187a9753b40f0927027adea8cebac` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:57.287082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a187a9753b40f0927027adea8cebac)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:57.289128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:57.410218 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a187a9753b40f0927027adea8cebac', 'request_id': 'req-1464983b-ccce-4942-9714-ed1b8f38bf82'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.417955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a187a9753b40f0927027adea8cebac', 'request_id': 'req-1464983b-ccce-4942-9714-ed1b8f38bf82'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.536221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a187a9753b40f0927027adea8cebac', 'request_id': 'req-1464983b-ccce-4942-9714-ed1b8f38bf82'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.547738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] The token cache is being invalidate because project b4a187a9753b40f0927027adea8cebac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:57.548027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.549002 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.550157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1117/2204] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:57 2022] DELETE /identity/v3/projects/b4a187a9753b40f0927027adea8cebac => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:57.554456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:57.565141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3716a8-202f-45aa-b511-16e5fde6fed1 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:57.633844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:57.634477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:57.634755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:57.635026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] PATH_INFO: `/v3/users/5dcd72fe876f40bc9fb4526e5d5c5c3e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:57.636139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dcd72fe876f40bc9fb4526e5d5c5c3e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:57.638929 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:57.797676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dcd72fe876f40bc9fb4526e5d5c5c3e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.798410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dcd72fe876f40bc9fb4526e5d5c5c3e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.805457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dcd72fe876f40bc9fb4526e5d5c5c3e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.806084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dcd72fe876f40bc9fb4526e5d5c5c3e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.868554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-80af469b-36fc-4c9f-b837-75947a4f56e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dcd72fe876f40bc9fb4526e5d5c5c3e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:57.886848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1088/2205] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:37:57 2022] DELETE /identity/v3/users/5dcd72fe876f40bc9fb4526e5d5c5c3e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:58.445958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:58.447481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:58.462911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464983b-ccce-4942-9714-ed1b8f38bf82 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:58.489357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:58.540337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:58.541591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:58.541591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:58.542086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] PATH_INFO: `/v3/projects/f59794bc82634507a4e89621989b34d9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:58.543167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59794bc82634507a4e89621989b34d9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:58.545824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:59.488017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59794bc82634507a4e89621989b34d9', 'request_id': 'req-eaf52552-d4e7-4780-bef5-0699a71afa04'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:59.494911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59794bc82634507a4e89621989b34d9', 'request_id': 'req-eaf52552-d4e7-4780-bef5-0699a71afa04'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:59.588377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59794bc82634507a4e89621989b34d9', 'request_id': 'req-eaf52552-d4e7-4780-bef5-0699a71afa04'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:59.605909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] The token cache is being invalidate because project f59794bc82634507a4e89621989b34d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:37:59.606316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:59.608035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaf52552-d4e7-4780-bef5-0699a71afa04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:37:59.620710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1118/2206] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:37:58 2022] DELETE /identity/v3/projects/f59794bc82634507a4e89621989b34d9 => generated 0 bytes in 1172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:37:59.622968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:59.623597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:59.623597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:59.636820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:59.662169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:59.663605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:59.750899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1089/2207] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:59.759918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:59.760293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:59.760650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:59.787710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:59.789358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:59.799989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1119/2208] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:37:59.804127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:59.817598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe4448-c5f3-47c9-ab1f-1a88eca6cc6b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:59.825002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:59.825002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:59.825002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:59.825002 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:59.826535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:37:59.826912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:59.828590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:59.841864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1090/2209] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:37:59.846635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:59.856072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e26e-b8d3-45ef-8355-f0f0e0d7d0e8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:59.861293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:59.861905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:59.862186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:59.862463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:59.863114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:59.865150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:59.955189 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1120/2210] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:37:59 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:37:59.959764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:59.970832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f951430-4aa3-4c21-bcf2-bcdc93a5bdba admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:00.041613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:00.042222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.042510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.042785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.043428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:00.045397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:00.143742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1091/2211] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:37:59 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:00.147766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:00.159571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606664ab-f227-4597-adf0-ac638f515253 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:00.164404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:00.165065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.165347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.165619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.166287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:00.167906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:00.307385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1121/2212] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:00.312130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:00.322140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d14968-1202-410a-a9fd-1e20a7da14ab admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:00.327806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f471297-6f1d-46f7-bc06-ba6c488d18a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:00.328397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f471297-6f1d-46f7-bc06-ba6c488d18a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.328701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f471297-6f1d-46f7-bc06-ba6c488d18a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.329014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f471297-6f1d-46f7-bc06-ba6c488d18a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.329679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f471297-6f1d-46f7-bc06-ba6c488d18a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:00.331386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f471297-6f1d-46f7-bc06-ba6c488d18a7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:00.342167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1092/2213] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:00.346253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:00.356424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f006e6d-0726-4e9a-82b2-2b6c72055527 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:00.361207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:00.361893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.362168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.362445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] PATH_INFO: `/v3/projects/175ee5addadf42488cc7384c0f2ef7a3/users/416ec048fffe48038b82a13eace491b8/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.379118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175ee5addadf42488cc7384c0f2ef7a3, user_id=416ec048fffe48038b82a13eace491b8, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:00.382601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:00.443820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1122/2214] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:00 2022] PUT /identity/v3/projects/175ee5addadf42488cc7384c0f2ef7a3/users/416ec048fffe48038b82a13eace491b8/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:00.454028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.454313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.454659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.491068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] MFA Rules not processed for user `416ec048fffe48038b82a13eace491b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:00.492322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:00.561878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:00.567207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1093/2215] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:00.573739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:00.630973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:00.643260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899aa29-2cc5-4b13-a8cd-d62271872ab2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:00.648203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:00.648809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.649121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.649395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.650178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:00.652590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:00.751212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1123/2216] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:00 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:00.756688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:00.780827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75de6-22f1-479a-92bd-6f2877a4ea27 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:00.937952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:00.937952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:00.937952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:00.937952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:00.938511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:00.938511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:01.039051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1094/2217] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:00 2022] POST /identity/v3/users => generated 297 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:38:01.043592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:01.056970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5227f-ffee-419e-ad08-714acea40a0f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:01.062424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:01.063079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.063562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.064056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.065301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:01.068041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:01.083478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1124/2218] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:01.091749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:01.103808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fcfe65-eb9b-45c7-9d59-86d34e46e9db admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:01.111355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:01.112063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.112351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.112629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] PATH_INFO: `/v3/projects/d57913008cb341108e85eddf290d30c2/users/e0679ac658eb40debef57d5e963b9874/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.137867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57913008cb341108e85eddf290d30c2, user_id=e0679ac658eb40debef57d5e963b9874, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:01.141809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:01.239205 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1095/2219] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:01 2022] PUT /identity/v3/projects/d57913008cb341108e85eddf290d30c2/users/e0679ac658eb40debef57d5e963b9874/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:01.240323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:01.251628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f3dfa-980e-4ed4-91a4-50e751165a46 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:01.337322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec355f-d667-4edf-842c-c163ecb421b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:01.338005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec355f-d667-4edf-842c-c163ecb421b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.338287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec355f-d667-4edf-842c-c163ecb421b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.338562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec355f-d667-4edf-842c-c163ecb421b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.339214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec355f-d667-4edf-842c-c163ecb421b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:01.340833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec355f-d667-4edf-842c-c163ecb421b6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:01.358444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:01.359511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1125/2220] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:01.381004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87854-9d2e-40db-8297-3d7e70090832 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:01.386303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:01.388392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.389053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.389679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] PATH_INFO: `/v3/projects/d57913008cb341108e85eddf290d30c2/users/e0679ac658eb40debef57d5e963b9874/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.392788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57913008cb341108e85eddf290d30c2, user_id=e0679ac658eb40debef57d5e963b9874, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:01.395908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:01.416928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1096/2221] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:01 2022] PUT /identity/v3/projects/d57913008cb341108e85eddf290d30c2/users/e0679ac658eb40debef57d5e963b9874/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:01.426995 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.427307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.427541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.468016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] MFA Rules not processed for user `e0679ac658eb40debef57d5e963b9874`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:01.469306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:01.591723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1126/2222] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:01.604170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:01.605978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:01.697629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6e5b7-0173-41c3-9c81-3a5183801c9a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:01.700449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:01.701871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.702155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.702466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.716802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:01.718526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:01.732431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1097/2223] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:01.899053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:01.910664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e76d6-2a39-471b-803a-4562a2822dd9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:01.915640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14ec3f-6f96-477a-b961-73f727738fa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:01.916924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14ec3f-6f96-477a-b961-73f727738fa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:01.917033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14ec3f-6f96-477a-b961-73f727738fa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:01.917519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14ec3f-6f96-477a-b961-73f727738fa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:01.985711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:02.000841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14ec3f-6f96-477a-b961-73f727738fa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:02.002430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14ec3f-6f96-477a-b961-73f727738fa1 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:02.043153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1127/2224] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:02.633077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:02.688692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:02.702861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599528c-0d58-4bb5-a72f-d4c518af5152 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:02.707376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:02.708011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:02.708288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:02.708557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] PATH_INFO: `/v3/users/416ec048fffe48038b82a13eace491b8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:02.709984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=416ec048fffe48038b82a13eace491b8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:02.712034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:02.953060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416ec048fffe48038b82a13eace491b8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:02.953811 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416ec048fffe48038b82a13eace491b8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:02.962288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416ec048fffe48038b82a13eace491b8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:02.962990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416ec048fffe48038b82a13eace491b8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:03.068160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416ec048fffe48038b82a13eace491b8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:03.092736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1098/2225] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:02 2022] DELETE /identity/v3/users/416ec048fffe48038b82a13eace491b8 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:03.094215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:03.094814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:03.094814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:03.115439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:03.118394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:03.185399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1128/2226] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:38:03.196751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:03.198365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:03.291188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f9132-da5f-4c26-9def-a7f682190073 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:03.306711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:03.307357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:03.307640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:03.307915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:03.321890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:03.323745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:03.335211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1099/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:03.975123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:03.977091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:03.991046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb3085-3cdc-4c4e-a4f7-0ae00afb2bbb None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:03.996057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:03.996699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:03.997019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:03.997394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] PATH_INFO: `/v3/projects/175ee5addadf42488cc7384c0f2ef7a3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:03.999039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=175ee5addadf42488cc7384c0f2ef7a3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:04.002983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:04.162918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175ee5addadf42488cc7384c0f2ef7a3', 'request_id': 'req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.169985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175ee5addadf42488cc7384c0f2ef7a3', 'request_id': 'req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.234405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175ee5addadf42488cc7384c0f2ef7a3', 'request_id': 'req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.249595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] The token cache is being invalidate because project 175ee5addadf42488cc7384c0f2ef7a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:04.249890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.250605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.252085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1129/2228] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:03 2022] DELETE /identity/v3/projects/175ee5addadf42488cc7384c0f2ef7a3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:04.256115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:04.266263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12cde-c79a-4969-9d00-ad8dac4deb6c service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:04.332413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:04.333043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:04.333322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:04.333594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] PATH_INFO: `/v3/users/e0679ac658eb40debef57d5e963b9874` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:04.335022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0679ac658eb40debef57d5e963b9874)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:04.336971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:04.353976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:04.514508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0679ac658eb40debef57d5e963b9874'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.515389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0679ac658eb40debef57d5e963b9874'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.524657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0679ac658eb40debef57d5e963b9874'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.526355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0679ac658eb40debef57d5e963b9874'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.600263 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fd1e6aea-d784-434b-bc64-9ce1fcf2b402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0679ac658eb40debef57d5e963b9874'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:04.648577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1100/2229] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:04 2022] DELETE /identity/v3/users/e0679ac658eb40debef57d5e963b9874 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:05.166821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.168512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:05.182341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedced3-2f1b-44b8-9540-7e6f7380dd3d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.240460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.241127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.241408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.241687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] PATH_INFO: `/v3/projects/d57913008cb341108e85eddf290d30c2` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.242818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57913008cb341108e85eddf290d30c2)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:05.244736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:05.255078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:05.344151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57913008cb341108e85eddf290d30c2', 'request_id': 'req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:05.350579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57913008cb341108e85eddf290d30c2', 'request_id': 'req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:05.432924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57913008cb341108e85eddf290d30c2', 'request_id': 'req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:05.444450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] The token cache is being invalidate because project d57913008cb341108e85eddf290d30c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:05.446312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:05.446312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b28bbd1e-0e0c-467b-9ebe-ec9aa6f9e517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:05.446944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1130/2230] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:05 2022] DELETE /identity/v3/projects/d57913008cb341108e85eddf290d30c2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:05.451999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.452518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.452518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.473569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:05.474770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:05.534455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1101/2231] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:38:05.542404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.542690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.542969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.563739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:05.565054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:05.573331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1131/2232] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:38:05.577934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.588148 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240311f-3e57-4cc2-ae2c-a98d5d53e7ee None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.593614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.594207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.594483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.594750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.595382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:38:05.595683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:05.597375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:05.609386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1102/2233] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:38:05.613788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.624268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb5b0c-5857-43f2-8946-54c129afe567 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.628989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.629719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.630000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.630297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.630939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:05.632956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:05.714755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1132/2234] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:05 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:05.719689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.729732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c12c8-f6b6-4a34-b3f1-02af81e133ef admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.786823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.787437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.787719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.787994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.788633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:05.790742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:05.880599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1103/2235] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:05 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:05.884729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.897287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa270d7-c5da-42b1-a2a2-d43a72a574d2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.912893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e33087-d038-458c-8463-98770972695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.912893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e33087-d038-458c-8463-98770972695f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.912893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e33087-d038-458c-8463-98770972695f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.912893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e33087-d038-458c-8463-98770972695f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.913478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e33087-d038-458c-8463-98770972695f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:05.913478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e33087-d038-458c-8463-98770972695f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:05.925281 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1133/2236] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:38:05.931981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.946320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220aac9-44e7-45a2-a551-62fc6bae8933 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.952347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ab767-40a9-46ff-8685-89ec7839700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.953266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ab767-40a9-46ff-8685-89ec7839700f admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.953622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ab767-40a9-46ff-8685-89ec7839700f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.953956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ab767-40a9-46ff-8685-89ec7839700f admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:05.954723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ab767-40a9-46ff-8685-89ec7839700f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:05.956899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ab767-40a9-46ff-8685-89ec7839700f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:05.965248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:05.971533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1104/2237] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:05.975934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e33087-d038-458c-8463-98770972695f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:05.991631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e33087-d038-458c-8463-98770972695f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:05.996897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:05.998365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:05.998716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:05.999070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] PATH_INFO: `/v3/projects/505f6f485b5f4d8489145f96714cc940/users/cf77c250e25449e3b12a3b61b666ed9d/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:06.014167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505f6f485b5f4d8489145f96714cc940, user_id=cf77c250e25449e3b12a3b61b666ed9d, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:06.016838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:06.109210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1134/2238] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:05 2022] PUT /identity/v3/projects/505f6f485b5f4d8489145f96714cc940/users/cf77c250e25449e3b12a3b61b666ed9d/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:38:06.114350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e69e2d-c1df-41b5-ace6-2f9f9539b363 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:06.114634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e69e2d-c1df-41b5-ace6-2f9f9539b363 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:06.114912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e69e2d-c1df-41b5-ace6-2f9f9539b363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:06.135424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c5e69e2d-c1df-41b5-ace6-2f9f9539b363 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:06.136763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c5e69e2d-c1df-41b5-ace6-2f9f9539b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:06.199863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1105/2239] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:06.210574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:06.212415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:06.274725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c0294-2477-4fdc-97b8-df5999beba27 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:06.279955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:06.280581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:06.280848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:06.281259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:06.290818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:06.292953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:06.302323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1135/2240] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:07.852207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:07.852757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:07.852757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:07.881326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] MFA Rules not processed for user `cf77c250e25449e3b12a3b61b666ed9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:07.882544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:07.958739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1106/2241] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:07.963339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:07.974015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d321d5-455f-4dbd-b910-6e78d1098194 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:07.978729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:07.979241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:07.979516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:07.979788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:07.980556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:07.982713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:08.102827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1136/2242] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:07 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:08.107350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:08.117863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b97c2-87ba-4573-be70-20546ec0cf22 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:08.174073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:08.174683 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:08.174959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:08.175239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:08.175868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:08.177831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:08.273493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1107/2243] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:08 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:08.277593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:08.288795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feef153-2726-4510-a9c9-b6615a22293b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:08.293537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:08.295166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:08.295459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:08.295735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:08.296381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:08.298009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:08.308937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1137/2244] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:08.314431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:08.324307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee9edf-f90f-4b54-9df6-1d8e85bfd869 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:08.329760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:08.330418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:08.330702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:08.330975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] PATH_INFO: `/v3/projects/edf5146257c7402bae4214ca78f6fe58/users/b716d72f2187477dafac1d68f872319d/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:08.347809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf5146257c7402bae4214ca78f6fe58, user_id=b716d72f2187477dafac1d68f872319d, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:08.350931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:08.354745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:08.434687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1108/2245] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:08 2022] PUT /identity/v3/projects/edf5146257c7402bae4214ca78f6fe58/users/b716d72f2187477dafac1d68f872319d/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:08.438689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:08.445195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:08.449040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ed27b-2358-4b38-ad1a-9587de8d4959 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:08.505008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c40783-985a-4250-9745-3de944113d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:08.505533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40783-985a-4250-9745-3de944113d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:08.505814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40783-985a-4250-9745-3de944113d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:08.506089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40783-985a-4250-9745-3de944113d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:08.506727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c40783-985a-4250-9745-3de944113d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:08.508322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c40783-985a-4250-9745-3de944113d50 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:08.518453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1138/2246] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:08.522603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:08.533522 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd578b-65b3-4dd9-a7e9-0d9001b40857 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:08.538161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:08.538810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:08.539091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:08.539367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] PATH_INFO: `/v3/projects/edf5146257c7402bae4214ca78f6fe58/users/b716d72f2187477dafac1d68f872319d/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:08.541334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf5146257c7402bae4214ca78f6fe58, user_id=b716d72f2187477dafac1d68f872319d, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:08.544043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:08.557028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1109/2247] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:08 2022] PUT /identity/v3/projects/edf5146257c7402bae4214ca78f6fe58/users/b716d72f2187477dafac1d68f872319d/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:09.937970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:09.938444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:09.938553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:09.968362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] MFA Rules not processed for user `b716d72f2187477dafac1d68f872319d`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:09.969707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:10.041036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1139/2248] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:10.052214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:10.053953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:10.110621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa0885-d7c5-4d94-a421-8a06fc1f4152 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:10.114215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02757405-0886-4f43-94fb-d3a54e899f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:10.114813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02757405-0886-4f43-94fb-d3a54e899f31 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:10.115091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02757405-0886-4f43-94fb-d3a54e899f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:10.115363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02757405-0886-4f43-94fb-d3a54e899f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:10.170083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02757405-0886-4f43-94fb-d3a54e899f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:10.171510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02757405-0886-4f43-94fb-d3a54e899f31 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:10.180490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1110/2249] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:11.808596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:11.810546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:11.822482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd760c-76cd-456a-9ee9-66609c4a174f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:11.826898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:11.827507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:11.827783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:11.828055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:11.837460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:11.838889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:11.848410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1140/2250] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:18.045711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:18.046162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:18.046255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:18.066045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:18.067328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:18.122037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1111/2251] 10.222.0.42 () {58 vars in 967 bytes} [Tue Oct 4 10:38:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:38:18.135116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:18.136501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:18.211252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb93f9-c2c7-45d2-bc84-b1b68279b547 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:18.218409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b6885-1506-4fca-9de6-b2a8c4f5f895 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:18.219015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6885-1506-4fca-9de6-b2a8c4f5f895 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:18.219289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6885-1506-4fca-9de6-b2a8c4f5f895 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:18.219562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6885-1506-4fca-9de6-b2a8c4f5f895 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:18.228812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b6885-1506-4fca-9de6-b2a8c4f5f895 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:18.230588 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b6885-1506-4fca-9de6-b2a8c4f5f895 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:18.238992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1141/2252] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:38:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:38:20.526560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:20.529769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:20.588918 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:20.620907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c81198-b88d-4949-b813-c98e0d0f0b63 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:20.628324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:20.628985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:20.629292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:20.629566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:20.631412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:20.643647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:20.645785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748cac8-a10f-417f-a08d-fa2307df35ba service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:20.657529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1112/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:27.620943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c7757-048c-4421-8d00-ed19499145b3 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:27.621924 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c7757-048c-4421-8d00-ed19499145b3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:27.622592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c7757-048c-4421-8d00-ed19499145b3 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:27.623360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1142/2254] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:38:27.635057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:27.635598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:27.635749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:27.658418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] MFA Rules not processed for user `a172b6b42d6443029ce492937796e2fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:27.659667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:27.670316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1113/2255] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:38:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:27.687600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c7757-048c-4421-8d00-ed19499145b3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:27.689479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-877c7757-048c-4421-8d00-ed19499145b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:27.699763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:27.706247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c7757-048c-4421-8d00-ed19499145b3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:27.711433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:27.712055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:27.712332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:27.712604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:27.720983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:27.722575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:27.732633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1143/2256] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:28.510092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:28.572902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b30e1f-5372-4c05-bc22-50ca97369661 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:28.577945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:28.578570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:28.578848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:28.579118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] PATH_INFO: `/v3/users/cf77c250e25449e3b12a3b61b666ed9d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:28.580235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf77c250e25449e3b12a3b61b666ed9d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:28.582179 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:28.826402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf77c250e25449e3b12a3b61b666ed9d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:28.827064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf77c250e25449e3b12a3b61b666ed9d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:28.834570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf77c250e25449e3b12a3b61b666ed9d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:28.835227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf77c250e25449e3b12a3b61b666ed9d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:28.913605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf77c250e25449e3b12a3b61b666ed9d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:28.932063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1114/2257] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:28 2022] DELETE /identity/v3/users/cf77c250e25449e3b12a3b61b666ed9d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:29.315332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:29.327574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb41517-385d-4a53-b6c0-973bb46f9398 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:29.399447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:29.400211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:29.400547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:29.400982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] PATH_INFO: `/v3/projects/505f6f485b5f4d8489145f96714cc940` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:29.402562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=505f6f485b5f4d8489145f96714cc940)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:29.405197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:29.507883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505f6f485b5f4d8489145f96714cc940', 'request_id': 'req-b72477c3-67eb-45f6-bc53-5e0cafda73ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:29.516488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505f6f485b5f4d8489145f96714cc940', 'request_id': 'req-b72477c3-67eb-45f6-bc53-5e0cafda73ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:29.620672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505f6f485b5f4d8489145f96714cc940', 'request_id': 'req-b72477c3-67eb-45f6-bc53-5e0cafda73ae'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:29.638336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] The token cache is being invalidate because project 505f6f485b5f4d8489145f96714cc940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:29.638725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:29.639868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:29.644184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1144/2258] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:29 2022] DELETE /identity/v3/projects/505f6f485b5f4d8489145f96714cc940 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:38:29.646365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:29.651932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:29.667693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fb29a-8ef8-44fc-900b-a65a16ec6a6e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:29.722666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:29.736288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:29.737055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:29.737405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:29.737750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] PATH_INFO: `/v3/users/b716d72f2187477dafac1d68f872319d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:29.738982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b716d72f2187477dafac1d68f872319d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:29.741253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:30.145766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b716d72f2187477dafac1d68f872319d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:30.145766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b716d72f2187477dafac1d68f872319d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:30.145766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b716d72f2187477dafac1d68f872319d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:30.145766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b716d72f2187477dafac1d68f872319d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:30.324520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-179cb8a2-ab44-4f11-a3b8-22b572ac021a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b716d72f2187477dafac1d68f872319d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:30.344538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1115/2259] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:29 2022] DELETE /identity/v3/users/b716d72f2187477dafac1d68f872319d => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:30.623423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:30.625569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:30.640778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72477c3-67eb-45f6-bc53-5e0cafda73ae admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:30.702836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:30.703461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:30.703743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:30.704039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] PATH_INFO: `/v3/projects/edf5146257c7402bae4214ca78f6fe58` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:30.705198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf5146257c7402bae4214ca78f6fe58)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:30.707162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:30.733728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:30.921101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf5146257c7402bae4214ca78f6fe58', 'request_id': 'req-068d2852-5d96-4938-a3a1-a2aa0ecdb503'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:30.927250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf5146257c7402bae4214ca78f6fe58', 'request_id': 'req-068d2852-5d96-4938-a3a1-a2aa0ecdb503'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:31.096107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf5146257c7402bae4214ca78f6fe58', 'request_id': 'req-068d2852-5d96-4938-a3a1-a2aa0ecdb503'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:31.107622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] The token cache is being invalidate because project edf5146257c7402bae4214ca78f6fe58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:31.107937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:31.108572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-068d2852-5d96-4938-a3a1-a2aa0ecdb503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:31.110048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1145/2260] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:30 2022] DELETE /identity/v3/projects/edf5146257c7402bae4214ca78f6fe58 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:31.114942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.115234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.115509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.136379 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:31.138182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:31.199214 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1116/2261] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:31.208053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.208423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.208764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.229049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:31.230260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:31.239162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1146/2262] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:31.243431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:31.253608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0336c-e6a6-459e-a0c3-58ea14cc1971 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:31.258201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:31.258790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.259071 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.259339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.259976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:38:31.260428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:31.262818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:31.274781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1117/2263] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:38:31.279935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:31.289167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede7de-93fc-4e21-b69f-195a05d366fe None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:31.294578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:31.295201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.295480 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.295753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.296383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:31.298330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:31.429295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1147/2264] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:31 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:31.433141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:31.443892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736dcb0-ab45-426d-9aea-e5d19723e5a1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:31.498848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:31.508494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:31.509145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.509422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.509695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.510330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:31.512377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:31.634135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1118/2265] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:31 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:31.653305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:31.664044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08fed5-e00b-4f63-9b97-451df135cfaf admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:31.669441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:31.670042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.670317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.670591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.671239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:31.673270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:31.685731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1148/2266] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:31.689937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:31.701436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fb876-eef0-4fb0-aa42-59abeb78dfc4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:31.706694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c0865-3cbf-41ce-916b-c19355018532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:31.707435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c0865-3cbf-41ce-916b-c19355018532 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.707721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c0865-3cbf-41ce-916b-c19355018532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.707995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c0865-3cbf-41ce-916b-c19355018532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.708642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c0865-3cbf-41ce-916b-c19355018532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:31.710568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c0865-3cbf-41ce-916b-c19355018532 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:31.721849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1119/2267] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:31.726050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:31.736733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3d155-0389-48da-9db3-62602546a30b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:31.741395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:31.742049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.742326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.742602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] PATH_INFO: `/v3/projects/2a119e6ae3714fb2b039bd566eac47e2/users/1b56d11f97db4365ae0d3a721a93c501/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.758355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a119e6ae3714fb2b039bd566eac47e2, user_id=1b56d11f97db4365ae0d3a721a93c501, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:31.761071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:31.850994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1149/2268] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:31 2022] PUT /identity/v3/projects/2a119e6ae3714fb2b039bd566eac47e2/users/1b56d11f97db4365ae0d3a721a93c501/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:38:31.863699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:31.864010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:31.864251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:31.894314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] MFA Rules not processed for user `1b56d11f97db4365ae0d3a721a93c501`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:31.895578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:31.962397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1120/2269] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:31.967511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.032964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18030e9-12fd-4262-84b5-330a03a20e01 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.037631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.038303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.038637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.038916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.039574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.041712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.135224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1150/2270] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:31 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:32.136534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.148646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf5a59-7d52-49e9-a16d-b828d5e93a2e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.207438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.208105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.208217 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.208494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.209168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.211421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.300479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1121/2271] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:32 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:32.301805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.312302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29a652-21d7-448f-8f9b-9ea72042f3ab admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.317185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.317792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.318069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.318339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.318977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.320585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.332605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1151/2272] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:32.337062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.348097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624669a2-4401-4cc1-a32d-2c27c73a69db admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.352743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.353481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.353765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.354042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] PATH_INFO: `/v3/projects/60d2b41c2d8d47ed9041304737bc732e/users/8c5f0eb4ad044325b9763a438b295160/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.372509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d2b41c2d8d47ed9041304737bc732e, user_id=8c5f0eb4ad044325b9763a438b295160, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.375218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.446168 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1122/2273] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:32 2022] PUT /identity/v3/projects/60d2b41c2d8d47ed9041304737bc732e/users/8c5f0eb4ad044325b9763a438b295160/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:32.450808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.461293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99db31-84ef-44a8-9232-2e29f2c8525f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.478220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:32.519527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be3629-f813-4443-b3a6-ad6b55f36f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.520140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be3629-f813-4443-b3a6-ad6b55f36f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.520428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be3629-f813-4443-b3a6-ad6b55f36f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.520700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be3629-f813-4443-b3a6-ad6b55f36f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.521383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be3629-f813-4443-b3a6-ad6b55f36f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.523101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be3629-f813-4443-b3a6-ad6b55f36f77 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.535694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1152/2274] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:32.537603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.548100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad6f6b-9909-467b-94ce-23ecc918746d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.553782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.558032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.558032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.558032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] PATH_INFO: `/v3/projects/60d2b41c2d8d47ed9041304737bc732e/users/8c5f0eb4ad044325b9763a438b295160/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.558032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d2b41c2d8d47ed9041304737bc732e, user_id=8c5f0eb4ad044325b9763a438b295160, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.560597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.571534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1123/2275] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:32 2022] PUT /identity/v3/projects/60d2b41c2d8d47ed9041304737bc732e/users/8c5f0eb4ad044325b9763a438b295160/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:32.582709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067dd4a-ded0-447c-8c5b-717bbf21bb4b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.582993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067dd4a-ded0-447c-8c5b-717bbf21bb4b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.583320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067dd4a-ded0-447c-8c5b-717bbf21bb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.612089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8067dd4a-ded0-447c-8c5b-717bbf21bb4b None None] MFA Rules not processed for user `8c5f0eb4ad044325b9763a438b295160`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:32.613379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8067dd4a-ded0-447c-8c5b-717bbf21bb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:32.688441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1153/2276] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:32.699831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:32.704265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:32.717562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:32.766386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0a302-af97-4c91-818f-63157934059f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:32.770055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:32.770672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:32.770955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:32.771232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:32.831235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:32.832690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:32.843319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1124/2277] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:34.950656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:34.951154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:34.951288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:34.971272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] MFA Rules not processed for user `961ed374bc664b299c1a6425dc803dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:34.972471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:35.029683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1154/2278] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:38.219208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:38.223029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:38.233495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149515c8-f1b6-4201-b8a0-1d02b2318204 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:38.237525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd065abd-2104-466d-9e70-37c92fedf6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:38.238209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065abd-2104-466d-9e70-37c92fedf6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:38.238567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065abd-2104-466d-9e70-37c92fedf6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:38.238842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065abd-2104-466d-9e70-37c92fedf6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:38.249079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd065abd-2104-466d-9e70-37c92fedf6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:38.250538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd065abd-2104-466d-9e70-37c92fedf6e1 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:38.260068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1125/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:44.380311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:44.427919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:44.442407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca0500-1e0a-426b-bc8a-5b58f163d7a0 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:44.446858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:44.447471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:44.447746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:44.448017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] PATH_INFO: `/v3/users/1b56d11f97db4365ae0d3a721a93c501` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:44.449288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b56d11f97db4365ae0d3a721a93c501)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:44.451712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:44.649908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b56d11f97db4365ae0d3a721a93c501'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:44.650573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b56d11f97db4365ae0d3a721a93c501'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:44.658243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b56d11f97db4365ae0d3a721a93c501'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:44.658944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b56d11f97db4365ae0d3a721a93c501'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:44.746666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b56d11f97db4365ae0d3a721a93c501'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:44.765498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1155/2280] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:44 2022] DELETE /identity/v3/users/1b56d11f97db4365ae0d3a721a93c501 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:44.770024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:44.770490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:44.770601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:44.789547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:44.790855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:44.862878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1126/2281] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:44.874790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:44.876828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:44.942200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf129-6bdd-4150-a09c-6a5e6b5c3d42 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:44.946425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:44.947044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:44.947310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:44.947583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:44.965860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:44.965860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:44.972008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1156/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:45.612238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:45.614178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:45.629779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cf88c-a6a9-4875-940d-d0dc206051f6 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:45.635334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:45.637481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:45.638388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:45.638755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] PATH_INFO: `/v3/projects/2a119e6ae3714fb2b039bd566eac47e2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:45.640292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a119e6ae3714fb2b039bd566eac47e2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:45.642262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:45.769463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:45.770997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a119e6ae3714fb2b039bd566eac47e2', 'request_id': 'req-964710ca-2692-46c8-8c60-ee7ee43a881d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:45.777250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a119e6ae3714fb2b039bd566eac47e2', 'request_id': 'req-964710ca-2692-46c8-8c60-ee7ee43a881d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:45.783447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a119e6ae3714fb2b039bd566eac47e2', 'request_id': 'req-964710ca-2692-46c8-8c60-ee7ee43a881d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:45.871170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] The token cache is being invalidate because project 2a119e6ae3714fb2b039bd566eac47e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:45.871170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:45.871170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:45.871170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1127/2283] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:45 2022] DELETE /identity/v3/projects/2a119e6ae3714fb2b039bd566eac47e2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:45.873447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:45.883246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d58aaf-d9fb-45d3-9f8a-8faf1e94b7c4 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:45.940787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:45.941478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:45.941754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:45.942026 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] PATH_INFO: `/v3/users/8c5f0eb4ad044325b9763a438b295160` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:45.943139 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5f0eb4ad044325b9763a438b295160)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:45.945084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:45.956824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:46.207578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5f0eb4ad044325b9763a438b295160'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:46.208239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5f0eb4ad044325b9763a438b295160'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:46.215461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5f0eb4ad044325b9763a438b295160'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:46.216292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5f0eb4ad044325b9763a438b295160'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:46.301715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5fe13f2f-7a74-4f1e-8125-8491a31d544b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5f0eb4ad044325b9763a438b295160'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:46.320922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1157/2284] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:45 2022] DELETE /identity/v3/users/8c5f0eb4ad044325b9763a438b295160 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:46.817544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:46.819022 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:46.833137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964710ca-2692-46c8-8c60-ee7ee43a881d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:46.896700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:46.897371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:46.897649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:46.897921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] PATH_INFO: `/v3/projects/60d2b41c2d8d47ed9041304737bc732e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:46.899070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d2b41c2d8d47ed9041304737bc732e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:46.901013 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.004601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d2b41c2d8d47ed9041304737bc732e', 'request_id': 'req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:47.010987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d2b41c2d8d47ed9041304737bc732e', 'request_id': 'req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:47.016843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d2b41c2d8d47ed9041304737bc732e', 'request_id': 'req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:47.089133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] The token cache is being invalidate because project 60d2b41c2d8d47ed9041304737bc732e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:47.089357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:47.090005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69c4ad74-760e-4b4e-8efe-dc979bd43c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:47.092147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1128/2285] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:46 2022] DELETE /identity/v3/projects/60d2b41c2d8d47ed9041304737bc732e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:47.098211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.098739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.098862 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.121609 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:47.122983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:47.182309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1158/2286] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:47.190837 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.191124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.191399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.212667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:47.213954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:47.222882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1129/2287] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:47.226880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.237076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cfb405-a6f2-4a55-a9f2-21b9e420cd68 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.241637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.242235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.242514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.242829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.243472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:38:47.243769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.245433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.257845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1159/2288] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:38:47.262240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.272650 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94813d56-8bf6-4c3f-ab7a-8fcb55fc4b69 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.277352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.277959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.278831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.279639 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.280293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.282249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.366528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1130/2289] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:47 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:47.370736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.381759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9b547-a743-4725-8cb6-ea3f1f221ad7 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.440565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.441221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.441529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.441807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.442451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.444432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.549229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1160/2290] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:47 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:38:47.553618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.565080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c52ac-19ae-45df-80a6-08393a148e6f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.570051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.575991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.575991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.575991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.575991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.575991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.585447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1131/2291] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:47.589785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.604086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:47.604086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862a67-ac6e-49ce-b2df-9a6d8c1a5407 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.606774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96d4b4-c617-4a65-9393-afa23391dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.607376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96d4b4-c617-4a65-9393-afa23391dda8 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.607655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96d4b4-c617-4a65-9393-afa23391dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.607932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96d4b4-c617-4a65-9393-afa23391dda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.608659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96d4b4-c617-4a65-9393-afa23391dda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.610315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96d4b4-c617-4a65-9393-afa23391dda8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.621631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1161/2292] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:38:47.626848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.636972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4446d95-d540-4797-9779-6c9802dea901 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.642335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.643099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.643384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.643661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] PATH_INFO: `/v3/projects/3f8a2d2dcce1489e8074d91e5e324f2b/users/b38e6a0fd92b4cc0bc5bb395fa6f25ac/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.659034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8a2d2dcce1489e8074d91e5e324f2b, user_id=b38e6a0fd92b4cc0bc5bb395fa6f25ac, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.661962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:47.742068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1132/2293] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:47 2022] PUT /identity/v3/projects/3f8a2d2dcce1489e8074d91e5e324f2b/users/b38e6a0fd92b4cc0bc5bb395fa6f25ac/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:47.755078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.755367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.755666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.784317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] MFA Rules not processed for user `b38e6a0fd92b4cc0bc5bb395fa6f25ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:47.785682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:47.852285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1162/2294] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:47.857499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:47.861885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:47.921630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de827313-8eba-4b55-baf1-d84656958876 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:47.926354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:47.927085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:47.927373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:47.927647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:47.928283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:47.930283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.016802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1133/2295] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:47 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:48.020973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.032250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2d5dc-818c-48cf-99a4-b2cea8b19f1d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.086383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.086982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.087258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.087528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.088155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.090178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.187209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1163/2296] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:48 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:38:48.191233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.202686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02010e7c-928d-4ba1-8bf6-3d4eb59e84d4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.207238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.207830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.208108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.208421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.209088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.210953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.222979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1134/2297] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:48.226950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.245727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45f4bf-7275-4a0c-b4d5-0442c3df300d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.251323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d0ede-ba3b-4d42-9620-5c8b049d71ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.251923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0ede-ba3b-4d42-9620-5c8b049d71ad admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.252206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0ede-ba3b-4d42-9620-5c8b049d71ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.252479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d0ede-ba3b-4d42-9620-5c8b049d71ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.253155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d0ede-ba3b-4d42-9620-5c8b049d71ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.254788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d0ede-ba3b-4d42-9620-5c8b049d71ad admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.265358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1164/2298] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:48.269574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.279986 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88522fbf-eb5e-4aa9-be15-5a67ecdd4bc8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.285353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.286023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.286301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.286573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] PATH_INFO: `/v3/projects/f7691e9c63404eb48d304b70fa934040/users/79ab3b16446a401ea66cb65b7038f754/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.300480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7691e9c63404eb48d304b70fa934040, user_id=79ab3b16446a401ea66cb65b7038f754, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.303136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.395475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1135/2299] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:48 2022] PUT /identity/v3/projects/f7691e9c63404eb48d304b70fa934040/users/79ab3b16446a401ea66cb65b7038f754/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:48.399763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.400052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.400330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.431993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] MFA Rules not processed for user `79ab3b16446a401ea66cb65b7038f754`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:48.433235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:48.483471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:48.504668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1165/2300] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:48.506273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.578459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f75a7-1c25-4d97-90f7-7c9b9853381d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.583308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.583793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.584132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.584329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.585341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.587115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.721220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1136/2301] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:38:48 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:38:48.725351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.736497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c800273-caa0-4c4d-bdb3-43413752f1f3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.794418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.795431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.795908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.796384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.797570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.801217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.902523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1166/2302] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:38:48 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:38:48.906868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.917665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e846a6-ec75-4020-9355-8722571143ed admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.922536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.923128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.923404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.923689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.924367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.925987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:48.936818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1137/2303] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:48.940961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:48.951709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666e4fb-4936-45c6-ab29-920099fe780f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:48.956320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:48.957055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:48.957344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:48.957635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] PATH_INFO: `/v3/projects/819f571c99e6422aa5f281d0b39ee074/users/d26e94bf145f4bc99783d8cfc32c9bad/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:48.973769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819f571c99e6422aa5f281d0b39ee074, user_id=d26e94bf145f4bc99783d8cfc32c9bad, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:48.976473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:49.050805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1167/2304] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:48 2022] PUT /identity/v3/projects/819f571c99e6422aa5f281d0b39ee074/users/d26e94bf145f4bc99783d8cfc32c9bad/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:49.054769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:49.065786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c158bb-2db8-4565-ad26-0c35d9e09bd9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:49.091632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:49.125687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e384cae-37d3-491b-830d-2f3681a9479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:49.126305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384cae-37d3-491b-830d-2f3681a9479d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:49.126588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384cae-37d3-491b-830d-2f3681a9479d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:49.126865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e384cae-37d3-491b-830d-2f3681a9479d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:49.127520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384cae-37d3-491b-830d-2f3681a9479d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:49.129238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e384cae-37d3-491b-830d-2f3681a9479d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:49.141305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1138/2305] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:38:49.145410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:49.156219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c871a-43a6-420c-a56b-c86645493364 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:49.161277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:49.162053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:49.162433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:49.162811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] PATH_INFO: `/v3/projects/819f571c99e6422aa5f281d0b39ee074/users/d26e94bf145f4bc99783d8cfc32c9bad/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:49.165023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=819f571c99e6422aa5f281d0b39ee074, user_id=d26e94bf145f4bc99783d8cfc32c9bad, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:49.167829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:49.179575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1168/2306] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:38:49 2022] PUT /identity/v3/projects/819f571c99e6422aa5f281d0b39ee074/users/d26e94bf145f4bc99783d8cfc32c9bad/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:49.191953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:49.192242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:49.192520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:49.222813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] MFA Rules not processed for user `d26e94bf145f4bc99783d8cfc32c9bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:49.224037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:49.301047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1139/2307] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:49.314607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:49.316335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:49.380277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb005277-8317-41c1-bac4-9f9ac012dd7a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:49.383968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:49.384587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:49.384882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:49.385172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:49.446511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:49.447952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:49.452030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:49.458030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1169/2308] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:51.463496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:51.489646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08679f0e-a1f7-4df3-a73e-ba862520491f None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:51.493365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:51.493981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:51.494262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:51.494533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:51.552424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:51.553886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:51.565218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1140/2309] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:53.494043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:53.503865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19f11b-131b-4d0d-8946-7647713af6cd service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:53.507579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8eb00-d74f-4a75-a67c-210856d931b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:53.508191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8eb00-d74f-4a75-a67c-210856d931b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:53.508443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8eb00-d74f-4a75-a67c-210856d931b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:53.508715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8eb00-d74f-4a75-a67c-210856d931b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:53.519610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8eb00-d74f-4a75-a67c-210856d931b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:53.521293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8eb00-d74f-4a75-a67c-210856d931b2 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:53.532580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1170/2310] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:58.133569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:58.180696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:58.197571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6ce71-64b2-40a3-80ce-8eec67087073 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:58.201160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:58.201838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:58.202122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:58.202409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] PATH_INFO: `/v3/users/b38e6a0fd92b4cc0bc5bb395fa6f25ac` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:58.203606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38e6a0fd92b4cc0bc5bb395fa6f25ac)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:58.205526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:58.424293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38e6a0fd92b4cc0bc5bb395fa6f25ac'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:58.425061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38e6a0fd92b4cc0bc5bb395fa6f25ac'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:58.432353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38e6a0fd92b4cc0bc5bb395fa6f25ac'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:58.433038 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38e6a0fd92b4cc0bc5bb395fa6f25ac'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:58.512776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38e6a0fd92b4cc0bc5bb395fa6f25ac'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:58.531078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1141/2311] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:58 2022] DELETE /identity/v3/users/b38e6a0fd92b4cc0bc5bb395fa6f25ac => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:58.535476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:58.535970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:58.536066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:58.556029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:38:58.557413 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:58.628516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1171/2312] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:38:58.638374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:58.639991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:58.703024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77c20e-f5df-45b3-8e66-3ad41da13064 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:58.706673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:58.707274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:58.707558 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:58.707833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:58.719145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:58.720980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:58.732853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1142/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:38:59.150036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:59.151682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:38:59.188605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088e5d1-ff14-4fc8-8f73-a01c96053339 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:59.193633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:59.194341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:59.194684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:59.195020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] PATH_INFO: `/v3/projects/3f8a2d2dcce1489e8074d91e5e324f2b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:59.196230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8a2d2dcce1489e8074d91e5e324f2b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:59.198335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:59.324182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8a2d2dcce1489e8074d91e5e324f2b', 'request_id': 'req-2f88330d-44cc-46a3-9acc-03d1610b5823'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.331229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8a2d2dcce1489e8074d91e5e324f2b', 'request_id': 'req-2f88330d-44cc-46a3-9acc-03d1610b5823'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.381291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8a2d2dcce1489e8074d91e5e324f2b', 'request_id': 'req-2f88330d-44cc-46a3-9acc-03d1610b5823'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.393711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] The token cache is being invalidate because project 3f8a2d2dcce1489e8074d91e5e324f2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:38:59.393999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.394684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.396348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1172/2314] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:38:59 2022] DELETE /identity/v3/projects/3f8a2d2dcce1489e8074d91e5e324f2b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:38:59.400103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:38:59.410691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b6d7c-b1a9-4882-8ea1-69af69612a81 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:38:59.470977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:38:59.471685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:38:59.471878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:38:59.472147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] PATH_INFO: `/v3/users/79ab3b16446a401ea66cb65b7038f754` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:38:59.473360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ab3b16446a401ea66cb65b7038f754)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:38:59.475301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:38:59.486099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:38:59.680424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ab3b16446a401ea66cb65b7038f754'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.681113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ab3b16446a401ea66cb65b7038f754'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.689014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ab3b16446a401ea66cb65b7038f754'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.689669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ab3b16446a401ea66cb65b7038f754'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.752606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ab3b16446a401ea66cb65b7038f754'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:38:59.771109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1143/2315] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:38:59 2022] DELETE /identity/v3/users/79ab3b16446a401ea66cb65b7038f754 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:39:00.165442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:00.169290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:00.181064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88330d-44cc-46a3-9acc-03d1610b5823 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:00.251119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:00.251760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:00.252042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:00.252318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] PATH_INFO: `/v3/projects/f7691e9c63404eb48d304b70fa934040` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:00.253561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7691e9c63404eb48d304b70fa934040)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:00.255606 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:00.266932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:00.389061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7691e9c63404eb48d304b70fa934040', 'request_id': 'req-a75a84e5-8038-475b-ae2c-58ac3b61344d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.396299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7691e9c63404eb48d304b70fa934040', 'request_id': 'req-a75a84e5-8038-475b-ae2c-58ac3b61344d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.455589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7691e9c63404eb48d304b70fa934040', 'request_id': 'req-a75a84e5-8038-475b-ae2c-58ac3b61344d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.467868 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] The token cache is being invalidate because project f7691e9c63404eb48d304b70fa934040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:39:00.468192 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.468913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.470491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1173/2316] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:39:00 2022] DELETE /identity/v3/projects/f7691e9c63404eb48d304b70fa934040 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:39:00.474487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:00.475930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:00.490132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca578a-feb5-4cf7-bc12-2a605bd3d9bc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:00.547828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:00.548464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:00.548769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:00.549067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] PATH_INFO: `/v3/users/d26e94bf145f4bc99783d8cfc32c9bad` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:00.550511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26e94bf145f4bc99783d8cfc32c9bad)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:00.552183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:00.749218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26e94bf145f4bc99783d8cfc32c9bad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.750027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26e94bf145f4bc99783d8cfc32c9bad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.758448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26e94bf145f4bc99783d8cfc32c9bad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.759243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26e94bf145f4bc99783d8cfc32c9bad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.815829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3b49d691-32d4-4614-a85d-c88deb9e14df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26e94bf145f4bc99783d8cfc32c9bad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:00.840433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1144/2317] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:39:00 2022] DELETE /identity/v3/users/d26e94bf145f4bc99783d8cfc32c9bad => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:39:01.571478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:01.573568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:01.588331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a84e5-8038-475b-ae2c-58ac3b61344d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:01.673998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:01.675249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:01.676185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:01.677831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] PATH_INFO: `/v3/projects/819f571c99e6422aa5f281d0b39ee074` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:01.679283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819f571c99e6422aa5f281d0b39ee074)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:01.681767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:01.783109 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819f571c99e6422aa5f281d0b39ee074', 'request_id': 'req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:01.791273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819f571c99e6422aa5f281d0b39ee074', 'request_id': 'req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:01.856767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819f571c99e6422aa5f281d0b39ee074', 'request_id': 'req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:01.869447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] The token cache is being invalidate because project 819f571c99e6422aa5f281d0b39ee074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:39:01.873201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:01.873952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-109a5b94-22d4-414f-a5b1-5f8fa673a2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:39:01.875809 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1174/2318] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:39:01 2022] DELETE /identity/v3/projects/819f571c99e6422aa5f281d0b39ee074 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:39:01.881627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:01.882486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:01.883146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:01.916258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:01.925656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:01.927176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:01.992721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1145/2319] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:02.001384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.001702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.001981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.026075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:02.027279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:02.036045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1175/2320] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:02.040337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.050381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0b44-d1e5-437f-9964-6bad58e98a83 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.054907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.055508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.055785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.056055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.056698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:39:02.057035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.058628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.070143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1146/2321] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:39:02.074178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.083532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7fc088-39bb-4397-9216-fdf8103fd246 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.088949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.089558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.089838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.090111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.091255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.093180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.164947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:02.171205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1176/2322] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:39:02 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:39:02.176010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.186005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4886a-391a-4b64-a803-5e8357ca822c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.244138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.244758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.245204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.245712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.246906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.250014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.330511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1147/2323] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:39:02 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:39:02.334954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.346625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443f494-9054-4730-af3c-d848295ac5bb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.351299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.351884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.352165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.352436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.353113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.354707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.364894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1177/2324] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:39:02.368880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.379628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22851c55-2515-4b07-9d0e-dd26de5fa2b1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.384231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe8426-dd10-4199-8577-5b1425a57d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.384819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8426-dd10-4199-8577-5b1425a57d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.385132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8426-dd10-4199-8577-5b1425a57d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.385405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe8426-dd10-4199-8577-5b1425a57d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.386046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe8426-dd10-4199-8577-5b1425a57d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.388622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe8426-dd10-4199-8577-5b1425a57d0a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.399140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1148/2325] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:39:02.403907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.414210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f457305-7672-410d-bdc3-e9bc6f27603f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.418911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.419551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.419831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.420107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] PATH_INFO: `/v3/projects/7d344597211d4f6b9a3c7961e7b0f706/users/f8d11882baae44968b8d451d6a30af87/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.434560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d344597211d4f6b9a3c7961e7b0f706, user_id=f8d11882baae44968b8d451d6a30af87, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.437312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.512142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1178/2326] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:39:02 2022] PUT /identity/v3/projects/7d344597211d4f6b9a3c7961e7b0f706/users/f8d11882baae44968b8d451d6a30af87/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:39:02.517716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f39aa-5e60-49b9-b263-244a6f266d92 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.518000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f39aa-5e60-49b9-b263-244a6f266d92 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.518273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f39aa-5e60-49b9-b263-244a6f266d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.537256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a80f39aa-5e60-49b9-b263-244a6f266d92 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:02.538456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a80f39aa-5e60-49b9-b263-244a6f266d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:02.597824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1149/2327] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:02.608628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:02.610486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:02.668647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2558443-2b7c-41d5-8957-43652ad60b44 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:02.672543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:02.673174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:02.673451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:02.673728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:02.682485 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:02.683930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:02.693988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1179/2328] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:39:04.187788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafcef-7a06-4f2a-8be5-a6d4a8d895fc None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:04.188334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafcef-7a06-4f2a-8be5-a6d4a8d895fc None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:04.188334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafcef-7a06-4f2a-8be5-a6d4a8d895fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:04.220843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:04.224355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b8fafcef-7a06-4f2a-8be5-a6d4a8d895fc None None] MFA Rules not processed for user `f8d11882baae44968b8d451d6a30af87`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:04.226017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b8fafcef-7a06-4f2a-8be5-a6d4a8d895fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:04.339172 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1150/2329] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:04.349028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:04.350212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:04.431732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed8c05-746d-4d23-b076-7d41e4a0e24b service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:04.436740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fee7a1-cc23-480d-a63b-3b669d29fef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:04.437821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fee7a1-cc23-480d-a63b-3b669d29fef0 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:04.438192 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fee7a1-cc23-480d-a63b-3b669d29fef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:04.438538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fee7a1-cc23-480d-a63b-3b669d29fef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:04.450616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fee7a1-cc23-480d-a63b-3b669d29fef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:04.452376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fee7a1-cc23-480d-a63b-3b669d29fef0 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:04.466592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1180/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:39:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:39:06.249907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc422-b69e-469c-b65f-05f24735195f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:06.250434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc422-b69e-469c-b65f-05f24735195f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:06.250535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128cc422-b69e-469c-b65f-05f24735195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:06.269446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-128cc422-b69e-469c-b65f-05f24735195f None None] MFA Rules not processed for user `f8d11882baae44968b8d451d6a30af87`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:06.270652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-128cc422-b69e-469c-b65f-05f24735195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:06.277441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1151/2331] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:07.698446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:07.699035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:07.699035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:07.720490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] MFA Rules not processed for user `c4f4844eb5e14d789476cfb44e2516f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:07.721735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:07.779962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1181/2332] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:10.647406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:10.647886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:10.647982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:10.670137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] MFA Rules not processed for user `3c7fcc99edab427aa2b750bf6b15b39c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:10.671969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:10.727919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1152/2333] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:39:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:17.771323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:20.721317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=54 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:04.313307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:04.314824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:04.327399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694e706-9078-418c-b813-bf3614d1a1d3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:04.330987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:04.331589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:04.331885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:04.332149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:04.334692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:04.411817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:04.413301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce96898-e999-49e1-9104-5ea5fdf5d8d7 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:04.425226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1182/2334] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:04.867968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:04.877380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396fb85-1277-47d6-adc6-0b30b6f930ef None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:04.882142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:04.882743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:04.883021 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:04.883292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:04.885058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:04.898711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:04.900176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa1310-e4ef-4df5-92a6-9b401121462e service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:04.911688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1153/2335] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:12.925050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:12.926528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:12.927681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:12.951396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] MFA Rules not processed for user `3008efcf02c644e3bc09850d6bd6eb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:12.952826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:13.031108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1183/2336] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:13.936142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:13.936777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:13.936777 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:13.956246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] MFA Rules not processed for user `961ed374bc664b299c1a6425dc803dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:13.957483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:14.017266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1154/2337] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:16.954640 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:16.956088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:16.974164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb5e1f-74e2-4247-be98-1f21f0e455d7 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:16.980005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:16.980614 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:16.980913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:16.981195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:16.983123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:16.994045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:16.995520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:17.005444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1184/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:19.457469 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:19.459895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:19.501940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:19.501940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:19.525348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aacf9-1000-440e-8fe5-8df4721af77a service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:19.538263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:19.538940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:19.539215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:19.539490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:19.541750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:19.554239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:19.555935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:19.557068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10dec1-4e12-4928-9e60-6de79348cd7e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:19.560799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:19.564141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:19.564141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:19.564141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:19.566973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1185/2339] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:40:19.571921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:19.578034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:19.597448 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1155/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:24.598089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:24.613278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2010fe5-8081-4555-9bf0-341410f59c40 service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:24.616988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:24.618007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:24.618007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:24.618179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:24.619965 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:24.632737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:24.634389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb2362-76cd-4bea-9f7b-f381a18cea80 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:24.643726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1186/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:25.030712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:25.042633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6755d5a-602c-41c6-9f99-dea463a4fb0e service glance] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:25.046280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:25.046883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:25.047165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:25.047467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:25.049230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:25.060304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:25.061770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:25.070819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1156/2342] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:32.444119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:32.444620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:32.444717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:32.451709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=78 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:32.464301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] MFA Rules not processed for user `f8d11882baae44968b8d451d6a30af87`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:32.465542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:32.472399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1187/2343] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:32.725843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:32.737160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=64 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:32.743880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb922df5-fd9d-4c08-864e-e6cf949f0b5f service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:32.747643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:32.748250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:32.748532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:32.748808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:32.752517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:32.763037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:32.764472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:32.775293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1157/2344] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:33.202886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:33.204691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:33.303774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82336e-61a6-44a3-9288-a782ad99ad21 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:33.309254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:33.309958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:33.310441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:33.310871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:33.312801 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:33.325981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:33.327573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:33.338929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1188/2345] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:34.011642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:34.021106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87475b-a2dc-408c-9b03-e8238ab4773b service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:34.025520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:34.026209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:34.026502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:34.026760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:34.028482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:34.040653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:34.042207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:34.053316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1158/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:38.089761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:38.117246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffc687-cd65-41e7-82fd-327e378eca53 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:38.120752 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:38.121559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:38.121841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:38.122105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:38.123822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:38.135641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:38.137483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:38.147846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1189/2347] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:40:39.398592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:39.400046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:39.420139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f8dfd-ff27-48a0-a8ca-c8ebad90fb2c service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:39.425403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:39.426642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:39.426919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:39.427188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] PATH_INFO: `/v3/users/c4f4844eb5e14d789476cfb44e2516f0` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:39.428319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f4844eb5e14d789476cfb44e2516f0)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:39.430321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:39.675750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f4844eb5e14d789476cfb44e2516f0'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:39.677110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f4844eb5e14d789476cfb44e2516f0'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:39.686139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f4844eb5e14d789476cfb44e2516f0'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:39.691049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f4844eb5e14d789476cfb44e2516f0'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:39.756212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-01e3c3f4-c1e7-4259-a632-2f6341434648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f4844eb5e14d789476cfb44e2516f0'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:39.775284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1159/2348] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:40:39 2022] DELETE /identity/v3/users/c4f4844eb5e14d789476cfb44e2516f0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:40.067888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:40.093846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4e638-07ff-4fb8-94de-c93df85f04cb service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:40.194418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:40.195821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:40.196153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:40.196436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] PATH_INFO: `/v3/projects/e94e5705bcfd476e87b2fa0af6ff597a` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:40.201911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94e5705bcfd476e87b2fa0af6ff597a)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:40.205802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:40.355022 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94e5705bcfd476e87b2fa0af6ff597a', 'request_id': 'req-124bc463-baba-4b32-bd47-42ae8b0317ce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:40.362635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94e5705bcfd476e87b2fa0af6ff597a', 'request_id': 'req-124bc463-baba-4b32-bd47-42ae8b0317ce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:40.464454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94e5705bcfd476e87b2fa0af6ff597a', 'request_id': 'req-124bc463-baba-4b32-bd47-42ae8b0317ce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:40.477961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] The token cache is being invalidate because project e94e5705bcfd476e87b2fa0af6ff597a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:40:40.478258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:40.478928 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-124bc463-baba-4b32-bd47-42ae8b0317ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:40.480423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1190/2349] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:40:40 2022] DELETE /identity/v3/projects/e94e5705bcfd476e87b2fa0af6ff597a => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:40.485733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:40.486233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:40.486337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:40.511047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:40.512614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:40.595202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1160/2350] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:40:40.602723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:40.603476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:40.603476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:40.615498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:40.639044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:40.639900 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:40.653592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1191/2351] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:40.655107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:40.668983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab3a1b-7293-4b86-99f5-804431d88729 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:40.674411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:40.675034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:40.675316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:40.675592 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:40.677787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:40:40.678157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:40.680328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:40.697703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1161/2352] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:40:40.706282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:40.717944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965f12-1feb-4d3d-943b-8675d6a837d2 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:40.726808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:40.729062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:40.729062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:40.729062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:40.729062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:40.730584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:40.846944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1192/2353] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:40:40 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:40.851624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:40.862657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebb4bd-f40a-4bea-b75f-0561cf3880eb admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:40.899656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:40.928070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:40.928815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:40.929224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:40.929606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:40.930384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:40.934061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:41.037202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1162/2354] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:40:40 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:41.038210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:41.048479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dbf55-5288-4c3e-9899-4ef027fdcb51 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:41.054616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:41.055406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:41.055787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:41.056177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:41.056902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:41.063766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:41.073260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1193/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:40:41.077935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:41.089153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1c37a-ba6a-4840-bd6f-488dee00e67c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:41.094457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fa37d-b790-40eb-ade4-a6a9e6698fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:41.095077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fa37d-b790-40eb-ade4-a6a9e6698fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:41.095356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fa37d-b790-40eb-ade4-a6a9e6698fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:41.095621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fa37d-b790-40eb-ade4-a6a9e6698fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:41.096298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fa37d-b790-40eb-ade4-a6a9e6698fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:41.098772 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fa37d-b790-40eb-ade4-a6a9e6698fb4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:41.113142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1163/2356] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:40:41.115952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:41.127055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6c1c9-09b3-43bb-a8e6-3de730ef9193 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:41.134199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:41.134879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:41.135159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:41.135433 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] PATH_INFO: `/v3/projects/0be389096ab5456f85c9f770a6bf1a42/users/0067953774cb4cabb34e5a3f7dd4e75c/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:41.153223 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be389096ab5456f85c9f770a6bf1a42, user_id=0067953774cb4cabb34e5a3f7dd4e75c, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:41.155911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:41.237212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1194/2357] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:41 2022] PUT /identity/v3/projects/0be389096ab5456f85c9f770a6bf1a42/users/0067953774cb4cabb34e5a3f7dd4e75c/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:41.249336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:41.249647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:41.249928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:41.281058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] MFA Rules not processed for user `0067953774cb4cabb34e5a3f7dd4e75c`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:41.282324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:41.356802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1164/2358] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:41.368462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:41.370483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:41.448536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bf27c-e0ca-4145-b3ab-07b7c4f9ad19 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:41.454179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00763a17-3f8f-4c4e-8782-1b0b0f39c575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:41.454720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00763a17-3f8f-4c4e-8782-1b0b0f39c575 service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:41.454999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00763a17-3f8f-4c4e-8782-1b0b0f39c575 service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:41.455273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00763a17-3f8f-4c4e-8782-1b0b0f39c575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:41.463506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00763a17-3f8f-4c4e-8782-1b0b0f39c575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:41.465131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00763a17-3f8f-4c4e-8782-1b0b0f39c575 service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:41.474435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1195/2359] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:41.690610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:41.757456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af868843-e316-42a9-8f02-fa44d8e1ac67 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:41.762209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:41.762843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:41.763124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:41.763474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] PATH_INFO: `/v3/users/0067953774cb4cabb34e5a3f7dd4e75c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:41.764611 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0067953774cb4cabb34e5a3f7dd4e75c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:41.766690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:41.956731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:41.959171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0067953774cb4cabb34e5a3f7dd4e75c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:41.959846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0067953774cb4cabb34e5a3f7dd4e75c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:41.967288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0067953774cb4cabb34e5a3f7dd4e75c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:41.967915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0067953774cb4cabb34e5a3f7dd4e75c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:42.036374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0067953774cb4cabb34e5a3f7dd4e75c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:42.058202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1165/2360] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:40:41 2022] DELETE /identity/v3/users/0067953774cb4cabb34e5a3f7dd4e75c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:42.062737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:42.063036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:42.063320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:42.082894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:42.088257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:42.131429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:42.133284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:42.163945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1196/2361] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:42.179512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:42.180901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:42.208788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:42.217854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37687bbb-6258-4dc5-b1e1-e5f589d94507 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:42.223983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:42.224796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:42.225193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:42.225547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:42.227297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:42.240697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c78bc-a263-40b5-92c0-3e744a76ce6e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:42.241155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:42.242959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4e5ee-6516-4f05-befd-85c7a49424cb service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:42.244424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:42.245087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:42.245370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:42.245698 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:42.254895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1166/2362] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:40:42.255741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:42.257341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:42.268082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1197/2363] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:40:42.792490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:42.794790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:42.794790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:42.814232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:42.815459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:42.869952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:42.871587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:42.902370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb0b86-60b3-45a8-9b46-a25212de561b service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:42.918067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:42.918902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:42.919331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:42.919930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] PATH_INFO: `/v3/projects/0be389096ab5456f85c9f770a6bf1a42` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:42.925306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be389096ab5456f85c9f770a6bf1a42)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:42.927582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1167/2364] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:40:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:42.928260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:43.187468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be389096ab5456f85c9f770a6bf1a42', 'request_id': 'req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:43.194817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be389096ab5456f85c9f770a6bf1a42', 'request_id': 'req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:43.210539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:43.211947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:43.265097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:43.283945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed872d-b9d6-4796-b20f-7775ecfbed12 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:43.299004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be389096ab5456f85c9f770a6bf1a42', 'request_id': 'req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:43.312503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] The token cache is being invalidate because project 0be389096ab5456f85c9f770a6bf1a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:40:43.313175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:43.314112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0d6825ec-595f-4cf0-a8ee-3cd50b73b3aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:43.320170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1198/2365] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:40:42 2022] DELETE /identity/v3/projects/0be389096ab5456f85c9f770a6bf1a42 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:40:43.321815 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f894b-41ce-403d-9734-8e998211b2c4 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.322275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f894b-41ce-403d-9734-8e998211b2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.322719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f894b-41ce-403d-9734-8e998211b2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.346975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-967f894b-41ce-403d-9734-8e998211b2c4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:43.348504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:43.348800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-967f894b-41ce-403d-9734-8e998211b2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:43.349226 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.349510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.349780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.351813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:43.415307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1199/2366] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:43.419550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:43.421210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:43.422991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.423277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.423709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.429917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1168/2367] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:40:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:40:43.444466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:43.445621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:43.456079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1200/2368] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:40:43.465310 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:43.482185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2f75-dabc-4e93-8a2d-c4f0a92476e1 service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:43.489056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:43.489814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.490091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.490388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.491009 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:40:43.491447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:43.493108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:43.518414 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1169/2369] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:40:43.520668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:43.534326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977fdd8-6c41-433f-a743-9fffd7ebb9f4 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:43.547706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:43.548127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.548430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.548918 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.549816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:43.552107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:43.734076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1201/2370] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:40:43 2022] POST /identity/v3/projects => generated 405 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:43.735535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:43.746852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30550b4-4231-4018-8a4f-c3866dd8b549 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:43.815035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:43.818519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.819553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.820020 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.820846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:43.823044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:43.971864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1170/2371] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:40:43 2022] POST /identity/v3/users => generated 303 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:43.973186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:43.985618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b757e4-6224-4d96-9c6a-5ece8b77e376 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:43.993214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:43.993841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:43.994122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:43.994401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:43.995048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:43.996672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:44.012825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1202/2372] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:40:44.014534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:44.034583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5f867-4449-4db9-98b8-4c85a0e30e9f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:44.039804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6544-a63f-40df-bbe0-61cdf00fbc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:44.040458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6544-a63f-40df-bbe0-61cdf00fbc1e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:44.040812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6544-a63f-40df-bbe0-61cdf00fbc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:44.041121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6544-a63f-40df-bbe0-61cdf00fbc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:44.041783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6544-a63f-40df-bbe0-61cdf00fbc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:44.043547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6544-a63f-40df-bbe0-61cdf00fbc1e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:44.059553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1171/2373] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:40:44.065501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:44.079667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0ed0-5a69-454b-a31d-1e412ad470ce admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:44.086123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:44.086969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:44.087316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:44.087684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] PATH_INFO: `/v3/projects/3f2811087b4a4115b1666241607bdeb6/users/e9b222c832f34a369626340986ad3533/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:44.108635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:44.123106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2811087b4a4115b1666241607bdeb6, user_id=e9b222c832f34a369626340986ad3533, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:44.126012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:44.218932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1203/2374] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:44 2022] PUT /identity/v3/projects/3f2811087b4a4115b1666241607bdeb6/users/e9b222c832f34a369626340986ad3533/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:44.224647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dacda-c95c-4e54-8d1f-39a5be85c792 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:44.225094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dacda-c95c-4e54-8d1f-39a5be85c792 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:44.226050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72dacda-c95c-4e54-8d1f-39a5be85c792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:44.236892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:44.251765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a72dacda-c95c-4e54-8d1f-39a5be85c792 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:44.253416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a72dacda-c95c-4e54-8d1f-39a5be85c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:44.342951 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1172/2375] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:44.355202 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:44.357123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:44.430953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db61d1-9967-4283-b53d-dd2fd581a5e9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:44.434853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:44.435465 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:44.435800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:44.436084 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:44.449736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:44.451204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:44.463466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1204/2376] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:46.032960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:46.033798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:46.034290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:46.074529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] MFA Rules not processed for user `e9b222c832f34a369626340986ad3533`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:46.075839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:46.156240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1173/2377] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:46 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:46.161394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:46.171969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a142b3c-5807-499d-8cf1-a12c8f6bfab3 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:46.176672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:46.177379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:46.177746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:46.178097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:46.178799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:46.181107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:46.272623 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1205/2378] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:40:46 2022] POST /identity/v3/projects => generated 405 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:46.273889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:46.285857 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a3aeb-e1e1-4188-9bec-10f1a51df106 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:46.350543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:46.351187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:46.351470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:46.351747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:46.352407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:46.354487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:46.734075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1174/2379] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:40:46 2022] POST /identity/v3/users => generated 309 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:46.739161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:46.749732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a08f23-340b-413e-bc41-61fe82c24627 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:46.755449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:46.755913 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:46.756212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:46.756494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:46.757167 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:46.758760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:46.775106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1206/2380] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:40:46.776452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:46.787042 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d443650-0609-4e09-91b5-df0c31196412 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:46.791785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:46.792457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:46.792741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:46.793058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] PATH_INFO: `/v3/projects/4d14afcd006b4d3580aaca58e29cf45f/users/f85d11912abc49df8da5651112a8acad/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:46.808648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d14afcd006b4d3580aaca58e29cf45f, user_id=f85d11912abc49df8da5651112a8acad, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:46.811293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:47.014725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:47.020988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1175/2381] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:46 2022] PUT /identity/v3/projects/4d14afcd006b4d3580aaca58e29cf45f/users/f85d11912abc49df8da5651112a8acad/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:47.024991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:47.036085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9952f-10cc-48d0-98b9-9793a1badd79 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:47.110179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd89a9b-7bd2-4885-95d0-81186e6b3acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:47.110792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd89a9b-7bd2-4885-95d0-81186e6b3acf admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:47.111075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd89a9b-7bd2-4885-95d0-81186e6b3acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:47.111351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd89a9b-7bd2-4885-95d0-81186e6b3acf admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:47.111996 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd89a9b-7bd2-4885-95d0-81186e6b3acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:47.114258 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd89a9b-7bd2-4885-95d0-81186e6b3acf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:47.126692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1207/2382] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:40:47.128950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:47.140451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29ddb0-4025-4648-a4e5-5a0c32829556 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:47.145919 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:47.146482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:47.146761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:47.147036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] PATH_INFO: `/v3/projects/4d14afcd006b4d3580aaca58e29cf45f/users/f85d11912abc49df8da5651112a8acad/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:47.148974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d14afcd006b4d3580aaca58e29cf45f, user_id=f85d11912abc49df8da5651112a8acad, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:47.151768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:47.178523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1176/2383] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:47 2022] PUT /identity/v3/projects/4d14afcd006b4d3580aaca58e29cf45f/users/f85d11912abc49df8da5651112a8acad/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:49.052047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:49.052530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:49.052745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:49.123808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] MFA Rules not processed for user `f85d11912abc49df8da5651112a8acad`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:49.126498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:49.239368 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:49.288545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1208/2384] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:51.460884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:51.600292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd134cb-172d-476f-9dc0-599ffb078f12 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:51.605305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:51.606296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:51.606574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:51.606843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] PATH_INFO: `/v3/users/e9b222c832f34a369626340986ad3533` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:51.607967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b222c832f34a369626340986ad3533)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:51.610429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:51.908818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b222c832f34a369626340986ad3533'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:51.910011 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b222c832f34a369626340986ad3533'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:51.921804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b222c832f34a369626340986ad3533'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:51.922826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b222c832f34a369626340986ad3533'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.013476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b222c832f34a369626340986ad3533'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.039301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1177/2385] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:40:51 2022] DELETE /identity/v3/users/e9b222c832f34a369626340986ad3533 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:52.336103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:52.365733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ada17-ed7e-41c4-9103-e07e34893ec4 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:52.452632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:52.453339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:52.453634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:52.453889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] PATH_INFO: `/v3/projects/3f2811087b4a4115b1666241607bdeb6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:52.455070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2811087b4a4115b1666241607bdeb6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:52.457295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:52.627282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2811087b4a4115b1666241607bdeb6', 'request_id': 'req-52f514fa-b824-449f-97ab-9c9db9c2e2a9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.644596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2811087b4a4115b1666241607bdeb6', 'request_id': 'req-52f514fa-b824-449f-97ab-9c9db9c2e2a9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.750848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2811087b4a4115b1666241607bdeb6', 'request_id': 'req-52f514fa-b824-449f-97ab-9c9db9c2e2a9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.763706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] The token cache is being invalidate because project 3f2811087b4a4115b1666241607bdeb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:40:52.764237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.765103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:52.767585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1209/2386] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:40:52 2022] DELETE /identity/v3/projects/3f2811087b4a4115b1666241607bdeb6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:52.771744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:52.774294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:52.799779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51493f07-c25e-447e-b595-e81bcbd1da53 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:52.876062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:52.876626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:52.876937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:52.877220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] PATH_INFO: `/v3/users/f85d11912abc49df8da5651112a8acad` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:52.879247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f85d11912abc49df8da5651112a8acad)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:52.881350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:52.890961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:53.141865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f85d11912abc49df8da5651112a8acad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:53.142551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f85d11912abc49df8da5651112a8acad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:53.157404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f85d11912abc49df8da5651112a8acad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:53.158239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f85d11912abc49df8da5651112a8acad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:53.236401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1dbefe4b-5684-46e4-8cd0-234656bbf500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f85d11912abc49df8da5651112a8acad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:53.266570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1178/2387] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:40:52 2022] DELETE /identity/v3/users/f85d11912abc49df8da5651112a8acad => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:53.571186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:53.572824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:53.595076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f514fa-b824-449f-97ab-9c9db9c2e2a9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:53.667904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:53.668528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:53.668844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:53.669193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] PATH_INFO: `/v3/projects/4d14afcd006b4d3580aaca58e29cf45f` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:53.670495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d14afcd006b4d3580aaca58e29cf45f)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:53.672494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:53.685671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:54.048545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d14afcd006b4d3580aaca58e29cf45f', 'request_id': 'req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:54.054721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d14afcd006b4d3580aaca58e29cf45f', 'request_id': 'req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:54.168196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d14afcd006b4d3580aaca58e29cf45f', 'request_id': 'req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:54.179473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] The token cache is being invalidate because project 4d14afcd006b4d3580aaca58e29cf45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:40:54.179757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:54.180385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3f7c4348-445c-4043-a8f9-4bb93c5db8d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:54.181892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1210/2388] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:40:53 2022] DELETE /identity/v3/projects/4d14afcd006b4d3580aaca58e29cf45f => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:54.186787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.187253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.187380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.207930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:54.209167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:54.278049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1179/2389] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:54.286763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.287051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.287324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.310471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:54.311759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:54.324132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1211/2390] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:54.325532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:54.336896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae734-bfe3-4ee2-bab1-d1a63659e1c2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:54.341654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:54.342292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.342577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.342854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.343508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:40:54.343812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:54.345544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:54.359115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1180/2391] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:40:54.363748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:54.373790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a9ebf-a0f4-4d5f-a797-8a422b56fe73 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:54.379346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:54.379957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.380238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.380514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.381191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:54.383203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:54.491108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1212/2392] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:40:54 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:54.497900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:54.511628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5524cac-1a60-4446-9d28-2246dfe681e4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:54.623504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:54.624544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.625061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.625493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.627034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:54.630428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:54.712991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:54.721750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1181/2393] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:40:54 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:54.724683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:54.734009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384381a-a7e9-43cf-ad68-5e98a5f9e95f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:54.739434 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:54.740037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.740319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.740594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.741299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:54.742941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:54.769328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:54.769746 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1213/2394] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:40:54.780810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81c79c-5227-4450-972e-2d775f7075c2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:54.787207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c9090-0257-43f2-893b-317879af9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:54.787816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c9090-0257-43f2-893b-317879af9054 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.788095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c9090-0257-43f2-893b-317879af9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.789564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c9090-0257-43f2-893b-317879af9054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.789564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c9090-0257-43f2-893b-317879af9054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:54.790994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c9090-0257-43f2-893b-317879af9054 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:54.807410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1182/2395] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:40:54.810319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:54.832197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beaaa8-f50a-4e13-b279-662ab1c45a65 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:54.838299 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:54.838979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:54.839257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:54.839533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] PATH_INFO: `/v3/projects/633793fbe65c4476ba07236c68f27727/users/5b7105aea1674eee99d3e78db40c2b54/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:54.858142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633793fbe65c4476ba07236c68f27727, user_id=5b7105aea1674eee99d3e78db40c2b54, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:54.860937 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.025034 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1214/2396] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:54 2022] PUT /identity/v3/projects/633793fbe65c4476ba07236c68f27727/users/5b7105aea1674eee99d3e78db40c2b54/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:55.036510 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.036792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.037108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.065155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] MFA Rules not processed for user `5b7105aea1674eee99d3e78db40c2b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:55.067439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:55.133769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1183/2397] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:55.138204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:55.200774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d64d5-3491-4ee1-8dc4-5cc57fe5daf0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:55.205382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:55.206012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.206323 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.206807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.207599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:55.209650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.364820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1215/2398] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:40:55 2022] POST /identity/v3/projects => generated 365 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:40:55.368803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:55.382063 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cac32-e6ea-4243-b3f2-491f70f6bbd4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:55.439255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:55.445717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.445717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.445717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.445717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:55.445717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.586117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1184/2399] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:40:55 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:40:55.590064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:55.600735 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21114689-0ae8-4a93-9456-cf84dfe5a4a2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:55.605219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:55.605811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.606087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.606381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.607007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:55.608603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.621878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1216/2400] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:40:55.625894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:55.640221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86391d-bf91-4060-88f3-548f05d1a436 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:55.652166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:55.653133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.654830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.654830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] PATH_INFO: `/v3/projects/c8118853ad4f4b05a9e08b2d1f4b423e/users/0f69445d62144d82a2f0305e69fc22f3/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.673294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8118853ad4f4b05a9e08b2d1f4b423e, user_id=0f69445d62144d82a2f0305e69fc22f3, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:55.676725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.791554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1185/2401] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:55 2022] PUT /identity/v3/projects/c8118853ad4f4b05a9e08b2d1f4b423e/users/0f69445d62144d82a2f0305e69fc22f3/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:55.796191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:55.806346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c4f30-967a-4684-9fbf-c8b429e356e2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:55.810904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:55.867213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e041f1d-7faa-45ed-892f-ea65bf1a3bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:55.867761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e041f1d-7faa-45ed-892f-ea65bf1a3bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.868042 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e041f1d-7faa-45ed-892f-ea65bf1a3bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.868342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e041f1d-7faa-45ed-892f-ea65bf1a3bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.869230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e041f1d-7faa-45ed-892f-ea65bf1a3bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:55.870710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e041f1d-7faa-45ed-892f-ea65bf1a3bb1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.881529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1217/2402] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:40:55.885680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:55.896571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8834a-a8f4-4de1-8de0-9ee8cb85685d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:55.901357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:55.902006 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.902283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.902581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] PATH_INFO: `/v3/projects/c8118853ad4f4b05a9e08b2d1f4b423e/users/0f69445d62144d82a2f0305e69fc22f3/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.904476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8118853ad4f4b05a9e08b2d1f4b423e, user_id=0f69445d62144d82a2f0305e69fc22f3, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:55.907155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:55.918329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1186/2403] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:40:55 2022] PUT /identity/v3/projects/c8118853ad4f4b05a9e08b2d1f4b423e/users/0f69445d62144d82a2f0305e69fc22f3/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:55.928709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:55.929029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:55.929312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:55.957415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] MFA Rules not processed for user `0f69445d62144d82a2f0305e69fc22f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:55.958655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:56.036378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1218/2404] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:56.047676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:56.049478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:56.084852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:56.108118 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecfd4a-f465-4540-96ba-c5b70a1440d1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:56.111719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405604-a82d-4b43-9aac-2e21905e02cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:56.112315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405604-a82d-4b43-9aac-2e21905e02cf service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:56.112636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405604-a82d-4b43-9aac-2e21905e02cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:56.112925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405604-a82d-4b43-9aac-2e21905e02cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:56.121799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91405604-a82d-4b43-9aac-2e21905e02cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:56.123301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91405604-a82d-4b43-9aac-2e21905e02cf service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:56.133152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1187/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:57.292717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:57.354731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6c2d2-b0c2-4e2f-ba48-09d4ed80c25f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:57.359183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:57.359798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:57.360074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:57.360342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] PATH_INFO: `/v3/users/5b7105aea1674eee99d3e78db40c2b54` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:57.361466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7105aea1674eee99d3e78db40c2b54)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:57.363812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:57.540945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:57.552886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7105aea1674eee99d3e78db40c2b54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:57.553550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7105aea1674eee99d3e78db40c2b54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:57.560359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7105aea1674eee99d3e78db40c2b54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:57.561019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7105aea1674eee99d3e78db40c2b54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:57.665081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7105aea1674eee99d3e78db40c2b54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:57.686787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1219/2406] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:40:57 2022] DELETE /identity/v3/users/5b7105aea1674eee99d3e78db40c2b54 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:57.691887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:57.692428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:57.692428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:57.712192 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:57.713421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:57.771643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1188/2407] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:57.781279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:57.782703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:57.840303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0b148-eb7f-4d89-be09-7b9f70d74098 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:57.843848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:57.844444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:57.844716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:57.845044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:57.853857 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:57.855282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:57.864985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1220/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:58.289481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:58.294814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:58.304164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317bc27-d23c-4986-9b7e-5253df3d22a9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:58.309177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:58.309821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:58.310431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:58.310431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] PATH_INFO: `/v3/projects/633793fbe65c4476ba07236c68f27727` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:58.311497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633793fbe65c4476ba07236c68f27727)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:58.313542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:58.448099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633793fbe65c4476ba07236c68f27727', 'request_id': 'req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.454221 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633793fbe65c4476ba07236c68f27727', 'request_id': 'req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.460084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633793fbe65c4476ba07236c68f27727', 'request_id': 'req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.550286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] The token cache is being invalidate because project 633793fbe65c4476ba07236c68f27727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:40:58.550571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.551184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.552736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1189/2409] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:40:58 2022] DELETE /identity/v3/projects/633793fbe65c4476ba07236c68f27727 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:58.556790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:58.567293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ce66a-0929-45b3-b1fd-d62302616ec3 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:58.625718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:58.626351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:58.626632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:58.626934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] PATH_INFO: `/v3/users/0f69445d62144d82a2f0305e69fc22f3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:58.628165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f69445d62144d82a2f0305e69fc22f3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:58.630332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:58.899249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f69445d62144d82a2f0305e69fc22f3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.900020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f69445d62144d82a2f0305e69fc22f3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.906468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:58.908873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f69445d62144d82a2f0305e69fc22f3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:58.910103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f69445d62144d82a2f0305e69fc22f3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.015178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7502049d-ade2-4701-9e8c-091c1ddf8d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f69445d62144d82a2f0305e69fc22f3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.037178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1221/2410] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:40:58 2022] DELETE /identity/v3/users/0f69445d62144d82a2f0305e69fc22f3 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:59.512341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:40:59.513904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:59.528832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb1faa-5cdc-4a20-bee2-e9352a6e16b4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:40:59.605581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:40:59.620210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:40:59.621256 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:59.621538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:59.621852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] PATH_INFO: `/v3/projects/c8118853ad4f4b05a9e08b2d1f4b423e` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:59.623667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8118853ad4f4b05a9e08b2d1f4b423e)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:40:59.625944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:40:59.758917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8118853ad4f4b05a9e08b2d1f4b423e', 'request_id': 'req-f4983bcc-2ab0-4285-ab2f-a824cce2189e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.765381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8118853ad4f4b05a9e08b2d1f4b423e', 'request_id': 'req-f4983bcc-2ab0-4285-ab2f-a824cce2189e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.771450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8118853ad4f4b05a9e08b2d1f4b423e', 'request_id': 'req-f4983bcc-2ab0-4285-ab2f-a824cce2189e'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.875587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] The token cache is being invalidate because project c8118853ad4f4b05a9e08b2d1f4b423e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:40:59.875881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.876565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f4983bcc-2ab0-4285-ab2f-a824cce2189e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:40:59.878136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1190/2411] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:40:59 2022] DELETE /identity/v3/projects/c8118853ad4f4b05a9e08b2d1f4b423e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:40:59.883845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:59.884341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:59.884458 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:40:59.905636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:40:59.906844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:40:59.968216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1222/2412] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:40:59.976561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:40:59.977400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:40:59.978208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.002199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:00.003410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:00.015484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1191/2413] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:00.020416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.030254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ecfd1e-5229-45d0-9b28-883f6c402c24 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.035879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.036493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.036773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.037082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.037728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:41:00.038032 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.039638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.053312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1223/2414] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:41:00.057997 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.067698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c635f-0532-4cc6-924c-bd96eb5b69fe None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.075304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.080415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.080415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.080415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.080415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.080415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.197570 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1192/2415] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:00 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:00.201969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.216903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae351e66-2154-4628-b571-083e0740edbc admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.278219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.278822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.279100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.279373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.280043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.282028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.397007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1224/2416] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:00 2022] POST /identity/v3/users => generated 294 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:00.400978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.411585 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fa811-905f-4035-af79-cb350089538d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.416206 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.416836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.417158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.417437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.424197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.424197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.430794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1193/2417] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:00.435118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.446164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8d322-5349-4f60-bfc4-8e1110fa886d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.450977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aad135-021c-41db-80ab-e5f56cc6981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.451592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad135-021c-41db-80ab-e5f56cc6981a admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.451859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad135-021c-41db-80ab-e5f56cc6981a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.457155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aad135-021c-41db-80ab-e5f56cc6981a admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.457155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aad135-021c-41db-80ab-e5f56cc6981a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.457155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aad135-021c-41db-80ab-e5f56cc6981a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.466415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1225/2418] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:00.470630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.480803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a4437-8b17-4b10-a9dd-328bf1e5d0a5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.485393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.486041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.486320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.486601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] PATH_INFO: `/v3/projects/7bb3535d12e24156b83177bafd007522/users/abd78b41dbe24b2789a002b75e6205c9/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.501088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb3535d12e24156b83177bafd007522, user_id=abd78b41dbe24b2789a002b75e6205c9, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.504324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.602758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1194/2419] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:00 2022] PUT /identity/v3/projects/7bb3535d12e24156b83177bafd007522/users/abd78b41dbe24b2789a002b75e6205c9/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:00.613955 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.614242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.614519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.646469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] MFA Rules not processed for user `abd78b41dbe24b2789a002b75e6205c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:00.647879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:00.676731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:00.718793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1226/2420] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:00.723843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.794154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a6ede-255b-424d-aa2b-9f37a2dfa3c0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.798888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.799497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.799781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.800964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.801607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.806634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.851357 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.854669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:00.926570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed4da36-dfc5-4d3e-847e-c439755853ed None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:00.932182 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:00.932786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:00.933105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:00.933381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:00.935113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:00.947219 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1195/2421] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:00 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:00.951201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:00.952282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:00.953993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:00.961557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:00.965510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1227/2422] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:41:01.016457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95ec86-3fe0-4762-9d28-37b1a83270e2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.021780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.022471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.022752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.023026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.023666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.025642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.139290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1196/2423] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:00 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:41:01.143394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:01.153974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676c550-6af5-4c1b-8bae-29665818b3db service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.158635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.159229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.160045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.160366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.161041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.162647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.172525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1228/2424] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:01.182397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:01.192619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24656aa-9d73-441b-a942-7f696e93f9d2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.197978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.203839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.203839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.203839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] PATH_INFO: `/v3/projects/b3f76ac4bc0d4812836547714c1438dc/users/0e1dee5ab1d34944b2be95a151cea7a2/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.214139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f76ac4bc0d4812836547714c1438dc, user_id=0e1dee5ab1d34944b2be95a151cea7a2, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.216905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.276257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1197/2425] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:01 2022] PUT /identity/v3/projects/b3f76ac4bc0d4812836547714c1438dc/users/0e1dee5ab1d34944b2be95a151cea7a2/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:01.280303 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:01.292001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2688549-8563-4a8c-8c13-a4c313bce513 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.350194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ebe6f-a4dd-40a2-9f07-2eb0ae132550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.350724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebe6f-a4dd-40a2-9f07-2eb0ae132550 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.351007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebe6f-a4dd-40a2-9f07-2eb0ae132550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.351283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebe6f-a4dd-40a2-9f07-2eb0ae132550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.351919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ebe6f-a4dd-40a2-9f07-2eb0ae132550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.353596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ebe6f-a4dd-40a2-9f07-2eb0ae132550 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.363860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1229/2426] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:01.368198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:01.378575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbc99b-355d-4a10-948a-652146ae75ac admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.383077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.383714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.383991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.384264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] PATH_INFO: `/v3/projects/b3f76ac4bc0d4812836547714c1438dc/users/0e1dee5ab1d34944b2be95a151cea7a2/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.386170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f76ac4bc0d4812836547714c1438dc, user_id=0e1dee5ab1d34944b2be95a151cea7a2, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.388937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.399122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1198/2427] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:01 2022] PUT /identity/v3/projects/b3f76ac4bc0d4812836547714c1438dc/users/0e1dee5ab1d34944b2be95a151cea7a2/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:01.409096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.409945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.410727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.438748 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] MFA Rules not processed for user `0e1dee5ab1d34944b2be95a151cea7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:01.439958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:01.466252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:01.507930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1230/2428] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:01.525894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:01.527660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:01.586106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86077-3117-437a-a26f-5b6d75855539 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.589831 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f19410-e4ee-4e3a-a4b4-7ae6a98413e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.590433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19410-e4ee-4e3a-a4b4-7ae6a98413e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.590709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19410-e4ee-4e3a-a4b4-7ae6a98413e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.590979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f19410-e4ee-4e3a-a4b4-7ae6a98413e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.600702 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19410-e4ee-4e3a-a4b4-7ae6a98413e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.602360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f19410-e4ee-4e3a-a4b4-7ae6a98413e4 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.612812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1199/2429] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:01.642882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:01.713089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda25fbf-37bd-47eb-8ad7-950bb6697be9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:01.717701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:01.718318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:01.718593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:01.718863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] PATH_INFO: `/v3/users/abd78b41dbe24b2789a002b75e6205c9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:01.719979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd78b41dbe24b2789a002b75e6205c9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:01.722666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:01.917554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd78b41dbe24b2789a002b75e6205c9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:01.918179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd78b41dbe24b2789a002b75e6205c9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:01.925277 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd78b41dbe24b2789a002b75e6205c9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:01.925952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd78b41dbe24b2789a002b75e6205c9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.013061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd78b41dbe24b2789a002b75e6205c9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.035332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1231/2430] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:01 2022] DELETE /identity/v3/users/abd78b41dbe24b2789a002b75e6205c9 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:02.038465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:02.038465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:02.038465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:02.056411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:02.057641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:02.117478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1200/2431] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:02.127150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:02.129244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:02.176360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:02.187765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68b404-60ae-428d-828a-fcc1fa1ad12f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:02.192647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:02.193268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:02.193547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:02.193823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:02.203184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:02.204648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:02.215246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1232/2432] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:02.616124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:02.618294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:02.627769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:02.632982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876563b7-1fe4-4177-ad3d-f8e076db290d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:02.637788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:02.638384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:02.638664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:02.639444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] PATH_INFO: `/v3/projects/7bb3535d12e24156b83177bafd007522` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:02.640761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb3535d12e24156b83177bafd007522)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:02.642784 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:02.744572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb3535d12e24156b83177bafd007522', 'request_id': 'req-d238da0b-19b8-441e-9412-b6fd6e878302'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.750801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb3535d12e24156b83177bafd007522', 'request_id': 'req-d238da0b-19b8-441e-9412-b6fd6e878302'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.757159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb3535d12e24156b83177bafd007522', 'request_id': 'req-d238da0b-19b8-441e-9412-b6fd6e878302'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.835017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] The token cache is being invalidate because project 7bb3535d12e24156b83177bafd007522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:02.835561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.836183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:02.837705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1201/2433] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:02 2022] DELETE /identity/v3/projects/7bb3535d12e24156b83177bafd007522 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:02.846562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:02.857029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195cdb-9ba0-4862-9fd5-da671c2f1faa service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:02.919172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:02.919802 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:02.920083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:02.920384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] PATH_INFO: `/v3/users/0e1dee5ab1d34944b2be95a151cea7a2` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:02.921550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1dee5ab1d34944b2be95a151cea7a2)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:02.923479 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:03.129268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1dee5ab1d34944b2be95a151cea7a2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.129947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1dee5ab1d34944b2be95a151cea7a2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.136877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1dee5ab1d34944b2be95a151cea7a2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.137531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1dee5ab1d34944b2be95a151cea7a2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.241642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-43542a18-2b54-42f4-8abe-172ea4a7c655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1dee5ab1d34944b2be95a151cea7a2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.258999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1233/2434] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:02 2022] DELETE /identity/v3/users/0e1dee5ab1d34944b2be95a151cea7a2 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:03.684050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:03.685496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:03.702753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238da0b-19b8-441e-9412-b6fd6e878302 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:03.779814 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:03.780533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:03.780813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:03.781112 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] PATH_INFO: `/v3/projects/b3f76ac4bc0d4812836547714c1438dc` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:03.782228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f76ac4bc0d4812836547714c1438dc)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:03.784196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:03.901800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f76ac4bc0d4812836547714c1438dc', 'request_id': 'req-2c3fecc8-bc09-445c-bb71-205a1313ea13'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.910257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f76ac4bc0d4812836547714c1438dc', 'request_id': 'req-2c3fecc8-bc09-445c-bb71-205a1313ea13'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:03.916269 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f76ac4bc0d4812836547714c1438dc', 'request_id': 'req-2c3fecc8-bc09-445c-bb71-205a1313ea13'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:04.038279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] The token cache is being invalidate because project b3f76ac4bc0d4812836547714c1438dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:04.038279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:04.038279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-2c3fecc8-bc09-445c-bb71-205a1313ea13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:04.039227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1202/2435] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:03 2022] DELETE /identity/v3/projects/b3f76ac4bc0d4812836547714c1438dc => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:04.044816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.045135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.045459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.068206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:04.069496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:04.143111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1234/2436] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:04.183654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.183654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.183846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.207333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:04.208567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:04.218001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1203/2437] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:04.222159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.234000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:04.243376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541d253-3d6d-4da0-b07c-3f98dd22752f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:04.249021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:04.249712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.250233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.250558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.251462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:41:04.251754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:04.258497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:04.277717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1235/2438] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:41:04.283706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.294924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc4956-97be-4e46-ba27-6f62e0c13b55 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:04.302257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:04.302972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.303253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.303530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.304397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:04.306871 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:04.317418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:04.478949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1204/2439] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:04 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:41:04.484739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.500446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f235c-ae6f-4148-b09c-75e207446301 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:04.574565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:04.575093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.575373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.575650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.576288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:04.578322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:04.723289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1236/2440] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:04 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:04.728012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.738098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cf0a6-01ee-4870-8b0e-758dbf676759 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:04.742670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:04.743285 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.743567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.743828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.744460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:04.746080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:04.764909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1205/2441] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:04.775720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.788958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14736866-7d94-44b4-9bf2-18f155327800 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:04.801420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dced9c-9718-4644-8b54-7e9d74f3bcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:04.802199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dced9c-9718-4644-8b54-7e9d74f3bcdb admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.802482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dced9c-9718-4644-8b54-7e9d74f3bcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.802752 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dced9c-9718-4644-8b54-7e9d74f3bcdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.803535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dced9c-9718-4644-8b54-7e9d74f3bcdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:04.805176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dced9c-9718-4644-8b54-7e9d74f3bcdb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:04.816734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1237/2442] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:04.820647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.830879 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b225b8-61e1-47fc-b46c-7c3e9d2fec8c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:04.837001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9694a7-fdb4-4dd9-bc5f-73a62e3c47ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:04.838586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9694a7-fdb4-4dd9-bc5f-73a62e3c47ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.838586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9694a7-fdb4-4dd9-bc5f-73a62e3c47ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.838586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9694a7-fdb4-4dd9-bc5f-73a62e3c47ab admin admin] PATH_INFO: `/v3/projects/acbd4f74bc3246c7a2238792b6810154/users/3c6afb51faec4b2f8eae9cb8a41b1e67/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.861029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9694a7-fdb4-4dd9-bc5f-73a62e3c47ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbd4f74bc3246c7a2238792b6810154, user_id=3c6afb51faec4b2f8eae9cb8a41b1e67, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:04.864270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9694a7-fdb4-4dd9-bc5f-73a62e3c47ab admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:04.879418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2706141-15cd-4a27-8f7e-dbbd94855189 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.879703 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2706141-15cd-4a27-8f7e-dbbd94855189 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.879979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2706141-15cd-4a27-8f7e-dbbd94855189 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.881613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1238/2443] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:41:04.886767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:04.887041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:04.887311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:04.908061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:04.913193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:04.979863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1239/2444] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:41:04.990711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1206/2445] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:04 2022] PUT /identity/v3/projects/acbd4f74bc3246c7a2238792b6810154/users/3c6afb51faec4b2f8eae9cb8a41b1e67/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:04.993404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:04.993404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.001872 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.002243 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.002547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.033246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] MFA Rules not processed for user `3c6afb51faec4b2f8eae9cb8a41b1e67`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:05.034678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.061063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a70a2-7ca1-4bb9-90ab-ecd2e9040fd2 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.075238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:05.103609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1207/2446] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:41:05.114324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.114555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.115063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.115342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.115618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.184410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c532-735f-4587-8d31-ebfdc1055281 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.189602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd8dc2-c6f0-429c-97a6-0a5093e2fb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.191946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.192576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd8dc2-c6f0-429c-97a6-0a5093e2fb7a admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.192576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd8dc2-c6f0-429c-97a6-0a5093e2fb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.192781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.192915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd8dc2-c6f0-429c-97a6-0a5093e2fb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.192915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd8dc2-c6f0-429c-97a6-0a5093e2fb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.194673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd8dc2-c6f0-429c-97a6-0a5093e2fb7a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.202675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1240/2447] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:05.334933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1208/2448] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:05 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:41:05.339111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.349386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a281f2-320d-4a79-8d11-be92e6a7f2b0 service glance] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.404846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.405404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.405682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.405957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.406582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.408494 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.471533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b787b-e13f-4a1e-8cdc-e7d2da1ea22f None None] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.472156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b787b-e13f-4a1e-8cdc-e7d2da1ea22f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.472156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b787b-e13f-4a1e-8cdc-e7d2da1ea22f None None] PATH_INFO: `/` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.473478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1209/2449] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:41:05.477741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.478019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.478290 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.494517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1241/2450] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:05 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:05.498666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.500433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] MFA Rules not processed for user `204544f0b58042c7957186df4bdd3949`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:05.501676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.510491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8fe97-cd2e-4eda-b50a-bffe0c7a7ea6 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.515358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.515951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.516228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.516503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.517171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.519119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.529343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:05.536568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1242/2451] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:41:05.536568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.544074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85151d63-4daa-45fa-8db5-dd95074734c0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.549070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effa945-7eea-4fef-bc77-b468851569b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.550362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effa945-7eea-4fef-bc77-b468851569b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.550502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effa945-7eea-4fef-bc77-b468851569b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.550780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effa945-7eea-4fef-bc77-b468851569b8 admin admin] PATH_INFO: `/v3/projects/5307105cb8034f638f7c9ffa21f315b3/users/77c9251207224f85a019d24e807592e7/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.561991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1210/2452] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:41:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:41:05.566885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effa945-7eea-4fef-bc77-b468851569b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5307105cb8034f638f7c9ffa21f315b3, user_id=77c9251207224f85a019d24e807592e7, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.573388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effa945-7eea-4fef-bc77-b468851569b8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.576039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.577464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.588128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4a75a-34ee-441f-a96d-1fedc55ac591 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.591868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.592467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.592748 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.593127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.600892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.602333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.610073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1211/2453] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:41:05.691821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1243/2454] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:05 2022] PUT /identity/v3/projects/5307105cb8034f638f7c9ffa21f315b3/users/77c9251207224f85a019d24e807592e7/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:05.692253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.695370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cc967-31d7-4b79-837e-e6bd25e3f302 None None] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.695655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cc967-31d7-4b79-837e-e6bd25e3f302 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.695926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cc967-31d7-4b79-837e-e6bd25e3f302 None None] PATH_INFO: `/` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.697335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1244/2455] 10.222.0.42 () {54 vars in 921 bytes} [Tue Oct 4 10:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:41:05.702080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.702381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.702661 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.704251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842027d-221b-4a54-bc81-c0bfff72e832 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.724227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] MFA Rules not processed for user `adc1b4c4a2de4d97bb8019c9efc05570`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:05.725493 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.734756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:05.770145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.770765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.771010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.771283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.771928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.773690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.785188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1212/2456] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:05.789439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.796774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1245/2457] 10.222.0.42 () {58 vars in 1018 bytes} [Tue Oct 4 10:41:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:41:05.800651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a7ce2-2c04-46da-ad72-6e26966d69d9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.805235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cde183-31d9-4e53-94d9-b36c87d5ae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.805890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cde183-31d9-4e53-94d9-b36c87d5ae7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.806232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cde183-31d9-4e53-94d9-b36c87d5ae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.806441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cde183-31d9-4e53-94d9-b36c87d5ae7a admin admin] PATH_INFO: `/v3/projects/5307105cb8034f638f7c9ffa21f315b3/users/77c9251207224f85a019d24e807592e7/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.808113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.808371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cde183-31d9-4e53-94d9-b36c87d5ae7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5307105cb8034f638f7c9ffa21f315b3, user_id=77c9251207224f85a019d24e807592e7, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:05.811029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cde183-31d9-4e53-94d9-b36c87d5ae7a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:05.821535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1213/2458] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:05 2022] PUT /identity/v3/projects/5307105cb8034f638f7c9ffa21f315b3/users/77c9251207224f85a019d24e807592e7/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:05.831835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.833094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.833094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.863262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] MFA Rules not processed for user `77c9251207224f85a019d24e807592e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:05.864480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.864823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a430-3cb6-4500-bd07-3189e8d7df2c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:05.915637 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:05.916168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:05.916446 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:05.916727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:05.934959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1214/2459] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:05.951970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:05.953405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:05.992643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:06.008953 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:06.010700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:06.016197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9bb7c-8e7a-4f5f-ad27-fa016ddd7848 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:06.021421 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d806c47-f976-4f16-ba35-f3a209aab625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:06.022752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d806c47-f976-4f16-ba35-f3a209aab625 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:06.022983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d806c47-f976-4f16-ba35-f3a209aab625 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:06.023296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d806c47-f976-4f16-ba35-f3a209aab625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:06.027982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1246/2460] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:41:06.039465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d806c47-f976-4f16-ba35-f3a209aab625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:06.040999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d806c47-f976-4f16-ba35-f3a209aab625 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:06.051402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1215/2461] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:41:06.123969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:06.130859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:06.202206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d81c7-9143-4463-a966-161f9307655d service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:06.207305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:06.208663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:06.208972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:06.209250 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] PATH_INFO: `/v3/users/3c6afb51faec4b2f8eae9cb8a41b1e67` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:06.210682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6afb51faec4b2f8eae9cb8a41b1e67)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:06.212745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:06.433217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6afb51faec4b2f8eae9cb8a41b1e67'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:06.433938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6afb51faec4b2f8eae9cb8a41b1e67'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:06.441012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6afb51faec4b2f8eae9cb8a41b1e67'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:06.441651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6afb51faec4b2f8eae9cb8a41b1e67'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:06.538972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6afb51faec4b2f8eae9cb8a41b1e67'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:06.558833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1247/2462] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:06 2022] DELETE /identity/v3/users/3c6afb51faec4b2f8eae9cb8a41b1e67 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:06.563906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:06.564188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:06.564462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:06.584154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:06.588820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:06.653023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1216/2463] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:06.663166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:06.664600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:06.722729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b790a71-556c-4a88-9872-07fe55608fce admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:06.726333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:06.726932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:06.727208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:06.727478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:06.736452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:06.738541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:06.748474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1248/2464] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:07.135497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:07.136988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:07.149747 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abe01e-a150-4f2b-91af-591615ef3efb None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:07.155105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:07.155756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:07.156024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:07.156296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] PATH_INFO: `/v3/projects/acbd4f74bc3246c7a2238792b6810154` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:07.157974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acbd4f74bc3246c7a2238792b6810154)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:07.163804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:07.260874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acbd4f74bc3246c7a2238792b6810154', 'request_id': 'req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.267302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acbd4f74bc3246c7a2238792b6810154', 'request_id': 'req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.273749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acbd4f74bc3246c7a2238792b6810154', 'request_id': 'req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.402610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] The token cache is being invalidate because project acbd4f74bc3246c7a2238792b6810154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:07.402905 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.403565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.406247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1217/2465] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:07 2022] DELETE /identity/v3/projects/acbd4f74bc3246c7a2238792b6810154 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:07.410755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:07.420963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690fd1f-5e55-4df7-97bb-1b6a496ca47a service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:07.440984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:07.477278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:07.477904 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:07.478186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:07.478448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] PATH_INFO: `/v3/users/77c9251207224f85a019d24e807592e7` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:07.479551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c9251207224f85a019d24e807592e7)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:07.481526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:07.713664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c9251207224f85a019d24e807592e7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.714339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c9251207224f85a019d24e807592e7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.721394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c9251207224f85a019d24e807592e7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.722476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c9251207224f85a019d24e807592e7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.790597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-6c80b45a-bf3e-48a8-bf68-703466c69faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c9251207224f85a019d24e807592e7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:07.810424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1249/2466] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:07 2022] DELETE /identity/v3/users/77c9251207224f85a019d24e807592e7 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:08.202970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:08.205252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:08.218703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8875a-45a0-4ccf-aba8-3ddae7aedd84 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:08.275487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:08.276110 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:08.276386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:08.276656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] PATH_INFO: `/v3/projects/5307105cb8034f638f7c9ffa21f315b3` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:08.277789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5307105cb8034f638f7c9ffa21f315b3)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:08.279680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:08.281084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:08.376391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5307105cb8034f638f7c9ffa21f315b3', 'request_id': 'req-f7b0573c-d33d-4ffe-b28a-5e907184eb40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:08.390058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5307105cb8034f638f7c9ffa21f315b3', 'request_id': 'req-f7b0573c-d33d-4ffe-b28a-5e907184eb40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:08.397123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5307105cb8034f638f7c9ffa21f315b3', 'request_id': 'req-f7b0573c-d33d-4ffe-b28a-5e907184eb40'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:08.499998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] The token cache is being invalidate because project 5307105cb8034f638f7c9ffa21f315b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:08.500401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:08.500975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f7b0573c-d33d-4ffe-b28a-5e907184eb40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:08.502562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1218/2467] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:08 2022] DELETE /identity/v3/projects/5307105cb8034f638f7c9ffa21f315b3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:08.508046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:08.508592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:08.508750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:08.530977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:08.532264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:08.596226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1250/2468] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:08.604782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:08.605499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:08.605861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:08.627126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:08.629555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:08.639356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1219/2469] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:08.644658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:08.655814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13207e35-97bd-4c9e-a12f-af232efe744b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:08.661411 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:08.662031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:08.662312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:08.662589 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:08.663324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:41:08.663634 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:08.665301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:08.677499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1251/2470] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:41:08.681892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:08.691800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbebf66-d468-4f35-bc5f-57e00b2d7b58 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:08.696677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:08.697431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:08.697710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:08.697984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:08.698615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:08.700534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:08.825886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1220/2471] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:08 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:08.829984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:08.841031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedea25-52e0-464b-8107-67c414f483f9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:08.869681 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:08.898173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:08.898780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:08.899064 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:08.902596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:08.902596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:08.902596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.019509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1252/2472] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:08 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:09.023583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.033738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e1e27-bf7d-4139-97b9-30c8f8e1afd1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.038476 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.039065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.039341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.039609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.040271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.041985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.052572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1221/2473] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:09.056520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.067908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13118b1-e8fe-43e9-8650-d92845160cdb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.073385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76e9c7-deca-4f41-a381-43befa29d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.074372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76e9c7-deca-4f41-a381-43befa29d25d admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.074737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76e9c7-deca-4f41-a381-43befa29d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.075069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76e9c7-deca-4f41-a381-43befa29d25d admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.075941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76e9c7-deca-4f41-a381-43befa29d25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.078298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76e9c7-deca-4f41-a381-43befa29d25d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.091193 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1253/2474] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:09.095337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.105724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b4c83-678e-4ac1-b3d1-93f0e2fb6e02 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.110949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.111666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.112034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.112947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] PATH_INFO: `/v3/projects/3ac2b6dc0ca945508dbc87c0559b420a/users/44f1674c59744ead8bcc78d5573a2331/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.129371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ac2b6dc0ca945508dbc87c0559b420a, user_id=44f1674c59744ead8bcc78d5573a2331, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.132029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.214200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1222/2475] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:09 2022] PUT /identity/v3/projects/3ac2b6dc0ca945508dbc87c0559b420a/users/44f1674c59744ead8bcc78d5573a2331/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:09.279512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.279796 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.280072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.308325 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] MFA Rules not processed for user `44f1674c59744ead8bcc78d5573a2331`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:09.309575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:09.373671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1254/2476] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:09.378481 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.446441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4b1e3-a712-4aba-bc39-9632c5434fab admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.451237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.451846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.452126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.452397 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.453061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.454981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.581404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1223/2477] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:09 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:09.586386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.599654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696d6eb-b3e9-411e-916e-f0c18e3243cd None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.672762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.673327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.673610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.673909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.674569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.677630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.807896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1255/2478] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:09 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:09.812122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.822539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5536ec6-e93d-4a2f-8c07-c4df3ce4c590 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.827149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.827729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.828016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.828400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.829328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.832088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.840073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:09.845484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1224/2479] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:41:09.850279 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.861161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292678d-7a61-4845-8ee0-3f5deda32c8c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:09.866587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:09.867375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:09.867658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:09.867935 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] PATH_INFO: `/v3/projects/4048fa816279416c9969aab2e11313bd/users/f4c1e4ec02b147e8824c9538fc3f6d54/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:09.883375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4048fa816279416c9969aab2e11313bd, user_id=f4c1e4ec02b147e8824c9538fc3f6d54, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:09.886251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:09.974241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1256/2480] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:09 2022] PUT /identity/v3/projects/4048fa816279416c9969aab2e11313bd/users/f4c1e4ec02b147e8824c9538fc3f6d54/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:09.978381 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:09.989312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c322b3c-7a11-4df3-b529-75733a855604 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:10.047627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea3501-2822-45e2-a2c1-0bfe3a3ef201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:10.048250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3501-2822-45e2-a2c1-0bfe3a3ef201 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.048531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3501-2822-45e2-a2c1-0bfe3a3ef201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.048801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3501-2822-45e2-a2c1-0bfe3a3ef201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.049486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea3501-2822-45e2-a2c1-0bfe3a3ef201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:10.051699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea3501-2822-45e2-a2c1-0bfe3a3ef201 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:10.061939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1225/2481] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:10.066685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:10.073578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:10.077859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407a353-4e73-4764-93e0-8f417dc43da9 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:10.084525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:10.085221 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.085754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.086029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] PATH_INFO: `/v3/projects/4048fa816279416c9969aab2e11313bd/users/f4c1e4ec02b147e8824c9538fc3f6d54/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.087947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4048fa816279416c9969aab2e11313bd, user_id=f4c1e4ec02b147e8824c9538fc3f6d54, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:10.090653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:10.102414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1257/2482] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:10 2022] PUT /identity/v3/projects/4048fa816279416c9969aab2e11313bd/users/f4c1e4ec02b147e8824c9538fc3f6d54/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:10.112962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.113511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.113847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.143193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] MFA Rules not processed for user `f4c1e4ec02b147e8824c9538fc3f6d54`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:10.145668 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:10.215945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1226/2483] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:10.226910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:10.228598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:10.286841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f760bb5-61a9-46ec-a9a6-a977734159e8 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:10.290498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddd7d4-c381-4ec1-b69e-12a20548a5e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:10.291107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddd7d4-c381-4ec1-b69e-12a20548a5e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.291441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddd7d4-c381-4ec1-b69e-12a20548a5e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.291656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddd7d4-c381-4ec1-b69e-12a20548a5e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.301080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddd7d4-c381-4ec1-b69e-12a20548a5e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:10.302618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddd7d4-c381-4ec1-b69e-12a20548a5e1 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:10.312039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1258/2484] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:10.418877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:10.428659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7435cf-9845-4f6e-a41f-c7c488f222cd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:10.432224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:10.432817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.433158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.433432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.492261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:10.493789 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:10.502651 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1227/2485] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:10.514138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.514717 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.514846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.534844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:10.536053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:10.593920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1259/2486] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:41:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:10.606284 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:10.607952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:10.635911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:10.690064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070e9e6-43cd-4e00-9fb6-3cc0c4d60183 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:10.695648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf93cc1-e9b6-414f-b68c-46a170b364b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:10.696249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93cc1-e9b6-414f-b68c-46a170b364b2 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:10.696526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93cc1-e9b6-414f-b68c-46a170b364b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:10.696817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93cc1-e9b6-414f-b68c-46a170b364b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:10.704695 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf93cc1-e9b6-414f-b68c-46a170b364b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:10.706405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf93cc1-e9b6-414f-b68c-46a170b364b2 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:10.714847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1228/2487] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:41:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:41:13.986624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:14.084060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8706e3a-022e-455c-9a9f-f1636a43b457 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:14.089287 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:14.089934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:14.090219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:14.090497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] PATH_INFO: `/v3/users/44f1674c59744ead8bcc78d5573a2331` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:14.091696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f1674c59744ead8bcc78d5573a2331)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:14.093711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:14.795505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:14.802208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f1674c59744ead8bcc78d5573a2331'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:14.802972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f1674c59744ead8bcc78d5573a2331'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:14.810951 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f1674c59744ead8bcc78d5573a2331'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:14.811798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f1674c59744ead8bcc78d5573a2331'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:14.939394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f1674c59744ead8bcc78d5573a2331'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:14.962842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1260/2488] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:13 2022] DELETE /identity/v3/users/44f1674c59744ead8bcc78d5573a2331 => generated 0 bytes in 974 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:14.964396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:14.964931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:14.965054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:14.985521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:14.986757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:15.050818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1229/2489] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:15.061417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:15.062920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:15.132399 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3f72a-d928-4235-9751-e9ee0dda490d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:15.136402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:15.137291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:15.137572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:15.137849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:15.149505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:15.152886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:15.173170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1261/2490] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:15.672553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:15.678908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:15.699927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3053b-20a0-48f5-af09-2dcbab1212c0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:15.705662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:15.706314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:15.706663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:15.707493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] PATH_INFO: `/v3/projects/3ac2b6dc0ca945508dbc87c0559b420a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:15.709224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ac2b6dc0ca945508dbc87c0559b420a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:15.711466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:15.861055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ac2b6dc0ca945508dbc87c0559b420a', 'request_id': 'req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:15.870126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ac2b6dc0ca945508dbc87c0559b420a', 'request_id': 'req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:15.901665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ac2b6dc0ca945508dbc87c0559b420a', 'request_id': 'req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:15.993072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] The token cache is being invalidate because project 3ac2b6dc0ca945508dbc87c0559b420a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:15.993072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:15.993786 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:15.994730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1230/2491] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:15 2022] DELETE /identity/v3/projects/3ac2b6dc0ca945508dbc87c0559b420a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:16.003098 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:16.014093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99add079-7101-4894-902e-2e36520c8435 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:16.078129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:16.078861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:16.079141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:16.079418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] PATH_INFO: `/v3/users/f4c1e4ec02b147e8824c9538fc3f6d54` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:16.080653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4c1e4ec02b147e8824c9538fc3f6d54)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:16.082715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:16.319443 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4c1e4ec02b147e8824c9538fc3f6d54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:16.320201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4c1e4ec02b147e8824c9538fc3f6d54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:16.328242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4c1e4ec02b147e8824c9538fc3f6d54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:16.329093 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4c1e4ec02b147e8824c9538fc3f6d54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:16.419007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a34e26aa-818b-4f34-b22e-84ddbb552577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4c1e4ec02b147e8824c9538fc3f6d54'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:16.420666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:16.445633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1262/2492] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:16 2022] DELETE /identity/v3/users/f4c1e4ec02b147e8824c9538fc3f6d54 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:16.992321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:16.993903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:17.021452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a36b5-e6c0-4a08-9ea3-6e5250d7dd45 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:17.070104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:17.110910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:17.112018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:17.112479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:17.113400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] PATH_INFO: `/v3/projects/4048fa816279416c9969aab2e11313bd` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:17.114882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4048fa816279416c9969aab2e11313bd)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:17.117450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:17.257812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4048fa816279416c9969aab2e11313bd', 'request_id': 'req-69fd89eb-cec4-4c1c-ac09-118e969888a5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:17.264405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4048fa816279416c9969aab2e11313bd', 'request_id': 'req-69fd89eb-cec4-4c1c-ac09-118e969888a5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:17.270543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4048fa816279416c9969aab2e11313bd', 'request_id': 'req-69fd89eb-cec4-4c1c-ac09-118e969888a5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:17.361574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] The token cache is being invalidate because project 4048fa816279416c9969aab2e11313bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:17.361793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:17.362490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-69fd89eb-cec4-4c1c-ac09-118e969888a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:17.364533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1231/2493] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:16 2022] DELETE /identity/v3/projects/4048fa816279416c9969aab2e11313bd => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:17.369496 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:17.370070 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:17.370180 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:17.405553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:17.406797 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:17.472540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1263/2494] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:17.481439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:17.481812 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:17.482174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:17.504444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:17.505820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:17.514633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1232/2495] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:41:17.519198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:17.529642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a541f9-3424-47b3-9c13-72fc330ce07b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:17.534826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:17.535500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:17.535771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:17.536051 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:17.536693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:41:17.537025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:17.538655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:17.555451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1264/2496] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:41:17.556891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:17.566806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f428b-b561-436d-bd8b-128e5a806cd2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:17.571828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:17.572613 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:17.572915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:17.579644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:17.579644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:17.579644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:17.746819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1233/2497] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:41:17 2022] POST /identity/v3/projects => generated 363 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:41:17.750658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:17.765178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596dbdf-2e98-4e8c-8a5e-cb5cdb0f9a1d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:17.864158 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:17.865141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:17.865604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:17.866031 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:17.866911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:17.869120 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:17.979619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1265/2498] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:41:17 2022] POST /identity/v3/users => generated 282 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:41:17.983783 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:17.995418 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a91ff-03fc-4e5a-ab44-63097ecb4c0f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:18.001199 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:18.001899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:18.002267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:18.002637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:18.003374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:18.005210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:18.019079 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1234/2499] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:18.020952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:18.031675 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf51a5-bc4d-4941-b6d5-e5d9197fbce6 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:18.039602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270fa5a3-00ec-436d-983b-58aa50c00c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:18.040220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa5a3-00ec-436d-983b-58aa50c00c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:18.040501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa5a3-00ec-436d-983b-58aa50c00c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:18.040778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270fa5a3-00ec-436d-983b-58aa50c00c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:18.041842 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270fa5a3-00ec-436d-983b-58aa50c00c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:18.043155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270fa5a3-00ec-436d-983b-58aa50c00c41 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:18.056896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1266/2500] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:41:18.060137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:18.071660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741d6b8-6a76-46f5-9b1a-7f74d55820fd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:18.076898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:18.077343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:18.077693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:18.077977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] PATH_INFO: `/v3/projects/712753ff3e6d4da593c7da7ee5fd20e8/users/213bad1a7cd24cd89cff4287012c3651/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:18.095312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712753ff3e6d4da593c7da7ee5fd20e8, user_id=213bad1a7cd24cd89cff4287012c3651, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:18.098850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:18.173273 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1235/2501] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:41:18 2022] PUT /identity/v3/projects/712753ff3e6d4da593c7da7ee5fd20e8/users/213bad1a7cd24cd89cff4287012c3651/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:41:18.185773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc57b5f-7282-4032-846b-da7f745552ab None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:18.186062 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc57b5f-7282-4032-846b-da7f745552ab None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:18.186342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc57b5f-7282-4032-846b-da7f745552ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:18.217266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1fc57b5f-7282-4032-846b-da7f745552ab None None] MFA Rules not processed for user `213bad1a7cd24cd89cff4287012c3651`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:18.218750 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1fc57b5f-7282-4032-846b-da7f745552ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:18.261581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:18.303282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1267/2502] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:41:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:18.311058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:18.318355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:18.415494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6510d8f-5086-4c3f-8458-3b370ec0c1d6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:18.421098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:18.422589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:18.422973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:18.423380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:18.438874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:18.440335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:18.455106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1236/2503] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:41:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:41:27.115348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:27.115877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:27.115877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:27.136515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:27.138088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:27.194416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1268/2504] 10.222.0.42 () {58 vars in 967 bytes} [Tue Oct 4 10:41:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:41:27.207647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:27.209852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:27.220332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:27.294181 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e9898-8fa2-4b99-b260-852dcc9d2cc1 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:27.301140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:27.302486 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:27.302839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:27.303188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:27.312005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:27.313825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:27.329277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1237/2505] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:41:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:41:37.186415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=36 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:37.338990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=31 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:57.998503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:58.000147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:58.077074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ba0d8-5702-413f-887b-dd5cd5d8f903 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:58.081817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:58.082474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:58.082753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:58.083027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] PATH_INFO: `/v3/users/f8d11882baae44968b8d451d6a30af87` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:58.084164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d11882baae44968b8d451d6a30af87)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:58.086467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:58.390610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d11882baae44968b8d451d6a30af87'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:58.391381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d11882baae44968b8d451d6a30af87'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:58.400445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d11882baae44968b8d451d6a30af87'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:58.401375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d11882baae44968b8d451d6a30af87'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:58.475420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d11882baae44968b8d451d6a30af87'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:58.525176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1269/2506] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:41:57 2022] DELETE /identity/v3/users/f8d11882baae44968b8d451d6a30af87 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:41:58.868163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:41:58.883697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26703c2a-e634-4cea-9367-484fc74cdb6e service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:41:58.958302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:41:58.958995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:58.959276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:58.959548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] PATH_INFO: `/v3/projects/7d344597211d4f6b9a3c7961e7b0f706` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:58.960939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d344597211d4f6b9a3c7961e7b0f706)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:41:58.963233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:41:59.116245 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d344597211d4f6b9a3c7961e7b0f706', 'request_id': 'req-725a3d6b-7a1a-4935-8785-601ea52cbd11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:59.123416 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d344597211d4f6b9a3c7961e7b0f706', 'request_id': 'req-725a3d6b-7a1a-4935-8785-601ea52cbd11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:59.129272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d344597211d4f6b9a3c7961e7b0f706', 'request_id': 'req-725a3d6b-7a1a-4935-8785-601ea52cbd11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:59.247612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] The token cache is being invalidate because project 7d344597211d4f6b9a3c7961e7b0f706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:41:59.247884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:59.248582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-725a3d6b-7a1a-4935-8785-601ea52cbd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:41:59.250236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1238/2507] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:41:58 2022] DELETE /identity/v3/projects/7d344597211d4f6b9a3c7961e7b0f706 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:03.814631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:03.816117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:03.830705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cd873-d199-42a4-94d3-b3b7d9e3b858 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:03.891663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:03.892310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:03.892595 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:03.892939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] PATH_INFO: `/v3/users/213bad1a7cd24cd89cff4287012c3651` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:03.894406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=213bad1a7cd24cd89cff4287012c3651)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:03.896909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:03.920670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:04.222178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '213bad1a7cd24cd89cff4287012c3651'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:04.222865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '213bad1a7cd24cd89cff4287012c3651'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:04.229874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '213bad1a7cd24cd89cff4287012c3651'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:04.230528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '213bad1a7cd24cd89cff4287012c3651'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:04.314923 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '213bad1a7cd24cd89cff4287012c3651'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:04.334055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1270/2508] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:03 2022] DELETE /identity/v3/users/213bad1a7cd24cd89cff4287012c3651 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:04.338909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:04.339404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:04.339512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:04.358601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:04.359801 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:04.417690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1239/2509] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:04.427336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:04.429035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:04.491111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f4b1e-7937-4fb3-b091-e4e709dd6510 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:04.495506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:04.496108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:04.496388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:04.496777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:04.505906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:04.507382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:04.518005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1271/2510] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:05.009693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:05.011336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:05.024597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b2308-a961-4791-944a-1ad9a22c50b2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:05.030377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:05.031196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:05.031477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:05.031751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] PATH_INFO: `/v3/projects/712753ff3e6d4da593c7da7ee5fd20e8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:05.032920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=712753ff3e6d4da593c7da7ee5fd20e8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:05.043435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:05.162485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:05.179354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712753ff3e6d4da593c7da7ee5fd20e8', 'request_id': 'req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:05.185689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712753ff3e6d4da593c7da7ee5fd20e8', 'request_id': 'req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:05.191123 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712753ff3e6d4da593c7da7ee5fd20e8', 'request_id': 'req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:05.276135 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] The token cache is being invalidate because project 712753ff3e6d4da593c7da7ee5fd20e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:05.276626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:05.277278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:05.293864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1240/2511] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:05 2022] DELETE /identity/v3/projects/712753ff3e6d4da593c7da7ee5fd20e8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:06.440402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:06.445121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:06.458028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c558d4d-9d7a-48eb-90ff-63a537dc869c service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:06.513831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:06.514439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:06.514715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:06.514987 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:06.516973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:06.582550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:06.595905 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:06.597400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:06.609140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1272/2512] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:10.115446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:10.115446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:10.142979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032dc7e-bac4-4cbf-a646-8df61ecd5f3a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:10.147981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:10.149561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:10.150045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:10.150767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:10.152691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:10.173225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:10.174713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:10.188649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1241/2513] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:10.916781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:10.919249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:10.939756 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651a213-2483-4e72-8bc5-0ca81c2cfc9c service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:10.946517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:10.947430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:10.947960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:10.948509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] PATH_INFO: `/v3/users/961ed374bc664b299c1a6425dc803dfa` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:10.950131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961ed374bc664b299c1a6425dc803dfa)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:10.952356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:11.213602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961ed374bc664b299c1a6425dc803dfa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:11.214297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961ed374bc664b299c1a6425dc803dfa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:11.222150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961ed374bc664b299c1a6425dc803dfa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:11.222841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961ed374bc664b299c1a6425dc803dfa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:11.332657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d766218d-5554-4641-8b76-a54b2b9c8f35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961ed374bc664b299c1a6425dc803dfa'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:11.357335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1273/2514] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:10 2022] DELETE /identity/v3/users/961ed374bc664b299c1a6425dc803dfa => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:11.819853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:11.830566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf10503-1bd7-486e-b8fb-fe2f02fae505 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:11.929776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:11.932852 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:11.933182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:11.933707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] PATH_INFO: `/v3/projects/29ede48c03e04fd6a00a874c318f5c3c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:11.935207 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ede48c03e04fd6a00a874c318f5c3c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:11.937755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:12.102532 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ede48c03e04fd6a00a874c318f5c3c', 'request_id': 'req-6605f1ac-c5d8-4d60-8f22-6d4bac592048'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:12.116719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ede48c03e04fd6a00a874c318f5c3c', 'request_id': 'req-6605f1ac-c5d8-4d60-8f22-6d4bac592048'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:12.124210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ede48c03e04fd6a00a874c318f5c3c', 'request_id': 'req-6605f1ac-c5d8-4d60-8f22-6d4bac592048'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:12.230332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] The token cache is being invalidate because project 29ede48c03e04fd6a00a874c318f5c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:12.230640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:12.231326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6605f1ac-c5d8-4d60-8f22-6d4bac592048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:12.237517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1242/2515] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:11 2022] DELETE /identity/v3/projects/29ede48c03e04fd6a00a874c318f5c3c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:12.239175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.239679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.239780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.262973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:12.264262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:12.369601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1274/2516] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:12.377401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.377694 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.384594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.399824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:12.401125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:12.411104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1243/2517] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:12.415850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:12.432963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b749c1b-e5fe-44a2-aea0-0d9a5f42df37 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:12.438719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:12.439702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.440200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.440700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.441713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:12.442237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:12.444089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:12.464205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1275/2518] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:12.466204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:12.475624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34456c-0949-4c02-bd63-818faa455053 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:12.481734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:12.482355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.482634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.482907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.483561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:12.485715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:12.619594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1244/2519] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:12 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:12.624196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:12.635136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ae640-d69d-457d-855a-01e28dac94dd admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:12.685363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:12.695707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:12.696407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.696695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.697000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.697635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:12.699558 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:12.876711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1276/2520] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:12 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:12.880708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:12.886604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:12.892608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7961da-af25-41b1-a1c6-e2870ad101bd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:12.899463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:12.900109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.900376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.900832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.901982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:12.904267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:12.918077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1245/2521] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:12.923565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:12.934546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd901bd-10b8-4f9e-9a4c-22be0cc73685 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:12.941929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26c3a-eef0-4389-a64b-498bb9e31ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:12.942635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26c3a-eef0-4389-a64b-498bb9e31ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.942911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26c3a-eef0-4389-a64b-498bb9e31ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.943190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26c3a-eef0-4389-a64b-498bb9e31ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:12.943865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd26c3a-eef0-4389-a64b-498bb9e31ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:12.946149 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd26c3a-eef0-4389-a64b-498bb9e31ee1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:12.962296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1277/2522] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:12.963912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:12.976432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7343166-cd14-4c68-8959-6b464bfcb279 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:12.982547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:12.983331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:12.983673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:12.984016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] PATH_INFO: `/v3/projects/8b0483abbf6b4988951239bc78ab5236/users/61fed0266f34458fb2fde04bacd01d67/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.134770 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0483abbf6b4988951239bc78ab5236, user_id=61fed0266f34458fb2fde04bacd01d67, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:13.137280 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:13.241220 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1246/2523] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:12 2022] PUT /identity/v3/projects/8b0483abbf6b4988951239bc78ab5236/users/61fed0266f34458fb2fde04bacd01d67/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:13.253991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:13.254568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:13.254867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.286365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] MFA Rules not processed for user `61fed0266f34458fb2fde04bacd01d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:13.287629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:13.358421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1278/2524] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:13.362975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:13.431854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773248ff-d6d2-4639-aa5a-15ecdbfb9f2a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:13.436550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:13.437185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:13.437463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:13.437734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.438358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:13.440321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:13.558238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1247/2525] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:13 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:13.560215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:13.572601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44e34a-dfce-4950-b9f6-8e99e9f1963f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:13.653171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:13.653817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:13.654097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:13.654371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.655009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:13.657007 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:13.802301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1279/2526] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:13 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:13.805776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:13.816391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93597-426e-4d1f-97aa-96ff621ad8e7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:13.821868 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:13.822278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:13.822552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:13.822817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.823459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:13.825262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:13.843015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1248/2527] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:13.846949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:13.858811 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85850d8-0644-48da-b98e-8192b140ecd0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:13.863680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376c068-6e2d-4967-b4cb-056372255823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:13.864345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376c068-6e2d-4967-b4cb-056372255823 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:13.864629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376c068-6e2d-4967-b4cb-056372255823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:13.864941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376c068-6e2d-4967-b4cb-056372255823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.865598 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376c068-6e2d-4967-b4cb-056372255823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:13.868709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376c068-6e2d-4967-b4cb-056372255823 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:13.890592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1280/2528] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:13.895461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:13.910563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc4b7-916a-4cb7-b658-32ce48123d0c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:13.915822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:13.916791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:13.917287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:13.917724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] PATH_INFO: `/v3/projects/7b5e4ffc498848888f04ae213fde94d2/users/1c03f8cce9e3491b8bfa21e1b256ffc5/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:13.935139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5e4ffc498848888f04ae213fde94d2, user_id=1c03f8cce9e3491b8bfa21e1b256ffc5, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:13.938204 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:14.047861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1249/2529] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:13 2022] PUT /identity/v3/projects/7b5e4ffc498848888f04ae213fde94d2/users/1c03f8cce9e3491b8bfa21e1b256ffc5/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:14.060040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:14.060335 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:14.060642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:14.092707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] MFA Rules not processed for user `1c03f8cce9e3491b8bfa21e1b256ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:14.094015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:14.181013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1281/2530] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:14.185654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:14.195388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:14.298635 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13766de8-ebec-4fea-9564-fa4cae6982ed admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:14.303366 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:14.304018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:14.304300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:14.304573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:14.305319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:14.307247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:14.532355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1250/2531] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:14 2022] POST /identity/v3/projects => generated 385 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:14.533833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:14.545798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c604a92-3aaf-4e4d-a344-e39c42e79d86 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:14.607817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:14.608415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:14.608692 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:14.608992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:14.609612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:14.611596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:14.682215 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:14.684762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:14.698409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30da12-78f8-47c9-bf12-cf6182f3dd83 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:14.703266 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:14.707785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:14.707785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:14.707785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] PATH_INFO: `/v3/users/3008efcf02c644e3bc09850d6bd6eb63` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:14.707785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3008efcf02c644e3bc09850d6bd6eb63)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:14.707785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:14.732958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:14.766087 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1282/2532] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:14 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:14.769053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:14.780322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72bc83-095c-45c7-bd53-08834f1552d7 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:14.785284 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:14.785879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:14.786152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:14.786419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:14.787052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:14.788660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:14.799068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1283/2533] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:14.803106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:14.813647 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55e4ac-ca9b-4cf7-9e56-fd2a9acea829 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:14.818488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:14.823459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:14.823459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:14.823459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] PATH_INFO: `/v3/projects/215c1e6668d54327be09ef84c2d48b02/users/d7700f5197c24fb0afea938b7dbe6041/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:14.835678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] RBAC: Authorizing `identity:create_grant(project_id=215c1e6668d54327be09ef84c2d48b02, user_id=d7700f5197c24fb0afea938b7dbe6041, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:14.838814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:14.995927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1284/2534] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:14 2022] PUT /identity/v3/projects/215c1e6668d54327be09ef84c2d48b02/users/d7700f5197c24fb0afea938b7dbe6041/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:14.999825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:15.032271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3008efcf02c644e3bc09850d6bd6eb63'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.033274 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3008efcf02c644e3bc09850d6bd6eb63'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.040845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3008efcf02c644e3bc09850d6bd6eb63'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.041544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3008efcf02c644e3bc09850d6bd6eb63'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.065129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b0cd-3530-4be4-bef5-afe82403845f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:15.069828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:15.070564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.071352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.071673 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.072252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:15.074037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:15.083561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1285/2535] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:42:15.088124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:15.089810 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.102003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899358-5c06-479e-a394-9f25775340e7 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:15.106730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4202cc-814b-4a69-aa7d-c639dc1508eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:15.107382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4202cc-814b-4a69-aa7d-c639dc1508eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.107700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4202cc-814b-4a69-aa7d-c639dc1508eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.107945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4202cc-814b-4a69-aa7d-c639dc1508eb admin admin] PATH_INFO: `/v3/projects/215c1e6668d54327be09ef84c2d48b02/users/d7700f5197c24fb0afea938b7dbe6041/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.109884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4202cc-814b-4a69-aa7d-c639dc1508eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=215c1e6668d54327be09ef84c2d48b02, user_id=d7700f5197c24fb0afea938b7dbe6041, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:15.112488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4202cc-814b-4a69-aa7d-c639dc1508eb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:15.123264 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1286/2536] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:15 2022] PUT /identity/v3/projects/215c1e6668d54327be09ef84c2d48b02/users/d7700f5197c24fb0afea938b7dbe6041/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:15.126122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3008efcf02c644e3bc09850d6bd6eb63'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.134590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f9579-b713-42ba-b287-462eb222e50f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.134877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f9579-b713-42ba-b287-462eb222e50f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.135157 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f9579-b713-42ba-b287-462eb222e50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.146440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1251/2537] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:14 2022] DELETE /identity/v3/users/3008efcf02c644e3bc09850d6bd6eb63 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:15.163946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-288f9579-b713-42ba-b287-462eb222e50f None None] MFA Rules not processed for user `d7700f5197c24fb0afea938b7dbe6041`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:15.165889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-288f9579-b713-42ba-b287-462eb222e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.235298 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1287/2538] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:15.247643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:15.249004 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.308627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fc1e6-0958-4f1e-952e-c5373cb0af99 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:15.312259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:15.312882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.313170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.313555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.315309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.375782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:15.377268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22bae9b-c9c7-4dc7-a7e5-1873fa4345da service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:15.386654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1252/2539] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:15.403097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f9579-b713-42ba-b287-462eb222e50f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:15.404551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-288f9579-b713-42ba-b287-462eb222e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.479352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f9579-b713-42ba-b287-462eb222e50f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:15.484039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:15.484695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.485168 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.485361 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] PATH_INFO: `/v3/projects/7857413ab9f24988bde94f9bf52b76a5` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.486762 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7857413ab9f24988bde94f9bf52b76a5)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:15.489165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:15.494121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:15.596260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7857413ab9f24988bde94f9bf52b76a5', 'request_id': 'req-f000af7d-d4f4-4480-b685-19a178c1438e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.603410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7857413ab9f24988bde94f9bf52b76a5', 'request_id': 'req-f000af7d-d4f4-4480-b685-19a178c1438e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.732096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7857413ab9f24988bde94f9bf52b76a5', 'request_id': 'req-f000af7d-d4f4-4480-b685-19a178c1438e'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.742643 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] The token cache is being invalidate because project 7857413ab9f24988bde94f9bf52b76a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:15.742921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.743563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f000af7d-d4f4-4480-b685-19a178c1438e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:15.745324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1288/2540] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:15 2022] DELETE /identity/v3/projects/7857413ab9f24988bde94f9bf52b76a5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:15.750603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.750924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.751178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.770588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:15.771765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.796291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:15.840196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1253/2541] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:15.846977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.847270 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.847741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.890100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:15.890100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:15.891024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1289/2542] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:15.894051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:15.906557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edcb00-31e1-4212-8f15-2f30ddeb2bfd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:15.911926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:15.912547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.912832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.913142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.913792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:15.914101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:15.916405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:15.930703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1254/2543] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:15.935825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:15.945835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c321-4c16-4934-8e3d-44e65e56f8d9 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:15.950768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:15.951513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:15.951860 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:15.952200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:15.952964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:15.955057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.093340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:16.095299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:16.153300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575191e-0c6e-4a00-9f46-13b3c9468d2a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:16.157142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:16.157681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.157965 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.158241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.160177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:16.173233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:16.174803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1290/2544] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:15 2022] POST /identity/v3/projects => generated 367 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:16.174971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.178917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:16.185378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1255/2545] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:16.245141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c79f-580d-4cdb-8aba-b4104e0ca2ee admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:16.249774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:16.250383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.250835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.251498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.253203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:16.254148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:16.256354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.372833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1291/2546] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:16 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:16.375883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:16.386959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a7654-dd2b-4eb4-8593-905f224aa653 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:16.393526 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:16.394400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.394829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.395111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.395763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:16.397431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.412017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1256/2547] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:16.413971 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:16.427853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa387ab9-85db-4229-8412-804d20614fc5 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:16.434220 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fba006-c3d3-4dd6-aede-8994a74354c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:16.434927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba006-c3d3-4dd6-aede-8994a74354c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.435274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba006-c3d3-4dd6-aede-8994a74354c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.435739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fba006-c3d3-4dd6-aede-8994a74354c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.436482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fba006-c3d3-4dd6-aede-8994a74354c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:16.438630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fba006-c3d3-4dd6-aede-8994a74354c9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.453138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1292/2548] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:42:16.457119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:16.467736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1076002-1c59-406b-8251-390014e0dd54 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:16.472575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:16.473375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.473661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.473937 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] PATH_INFO: `/v3/projects/bd90fe960dd44e569d79910393cd5556/users/94a7b98e02ab4d699248258e44e21856/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.498097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd90fe960dd44e569d79910393cd5556, user_id=94a7b98e02ab4d699248258e44e21856, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:16.498097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.605996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1257/2549] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:16 2022] PUT /identity/v3/projects/bd90fe960dd44e569d79910393cd5556/users/94a7b98e02ab4d699248258e44e21856/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:16.613743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.614300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.614300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.646617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] MFA Rules not processed for user `94a7b98e02ab4d699248258e44e21856`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:16.647991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:16.730260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1293/2550] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:16.742878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:16.746127 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:16.809863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ea285-31c5-4b0c-bd80-3f10af3aa452 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:16.813730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:16.814342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:16.814624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:16.814897 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:16.824957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:16.830860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:16.835791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1258/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:17.747196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:17.747196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:17.758163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efbfbe-18be-4a42-b349-1924c9627a80 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:17.761700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:17.762300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:17.762574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:17.762844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:17.764511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:17.826569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:17.828046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:17.831662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:17.838289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1294/2552] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:19.727932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:19.737490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d26b6-ceaa-4451-890d-d503da245992 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:19.762437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:19.762762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:19.762983 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:19.763264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:19.765157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:19.818873 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:19.865519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:19.866851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:19.880884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1259/2553] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:20.353176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:20.355330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:20.435055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de3b8f-58b2-4f5d-b7c8-58526eb86e53 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:20.440063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:20.440666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:20.440975 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:20.441252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:20.442974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:20.455320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:20.456839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:20.475972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1295/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:20.476732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:20.479353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:20.551182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe846a-c8bb-490e-a4e6-38ace787bec4 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:20.556411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:20.557301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:20.557769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:20.558386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] PATH_INFO: `/v3/users/3c7fcc99edab427aa2b750bf6b15b39c` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:20.559719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7fcc99edab427aa2b750bf6b15b39c)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:20.562456 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:20.770144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7fcc99edab427aa2b750bf6b15b39c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:20.770878 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7fcc99edab427aa2b750bf6b15b39c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:20.778404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7fcc99edab427aa2b750bf6b15b39c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:20.779031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7fcc99edab427aa2b750bf6b15b39c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:20.891779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-c7766f81-caba-4534-85e8-e2b946a80ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7fcc99edab427aa2b750bf6b15b39c'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:20.909619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1260/2555] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:20 2022] DELETE /identity/v3/users/3c7fcc99edab427aa2b750bf6b15b39c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:21.203237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:21.220844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb53300-a3c2-4a85-ba75-e94b1c5306b5 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:21.281699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:21.282337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:21.287536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:21.287536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] PATH_INFO: `/v3/projects/17e605186f7944a79d36288d99d1eaa9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:21.287536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17e605186f7944a79d36288d99d1eaa9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:21.287536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:21.420684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17e605186f7944a79d36288d99d1eaa9', 'request_id': 'req-77c2ba3d-1477-40fa-a78c-8934948d6818'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:21.427363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17e605186f7944a79d36288d99d1eaa9', 'request_id': 'req-77c2ba3d-1477-40fa-a78c-8934948d6818'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:21.556154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17e605186f7944a79d36288d99d1eaa9', 'request_id': 'req-77c2ba3d-1477-40fa-a78c-8934948d6818'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:21.571337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] The token cache is being invalidate because project 17e605186f7944a79d36288d99d1eaa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:21.571337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:21.571337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-77c2ba3d-1477-40fa-a78c-8934948d6818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:21.571337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1296/2556] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:21 2022] DELETE /identity/v3/projects/17e605186f7944a79d36288d99d1eaa9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:21.576165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:21.576450 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:21.576720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:21.597201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:21.598407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:21.636143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:21.661156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1261/2557] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:21.670476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:21.670701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:21.670981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:21.690556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:21.691779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:21.699820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1297/2558] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:21.704077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:21.714507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a1a15-e3b5-47c2-ad93-1ddf0becb1dd None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:21.720704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:21.726313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:21.726313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:21.726313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:21.726313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:21.726313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:21.726313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:21.740089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1262/2559] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:21.744946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:21.754635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f83c0-0c05-4d0b-ba54-1df6b4fe8f61 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:21.760176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:21.760871 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:21.761161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:21.761435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:21.762076 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:21.764071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:21.775628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:21.913976 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1298/2560] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:21 2022] POST /identity/v3/projects => generated 401 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:21.917105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:21.926567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03ad5d-39f0-4f04-b7cb-8c3d0cd33081 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:21.984461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:21.985096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:21.985400 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:21.985672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:21.986304 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:21.988233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.149711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1263/2561] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:21 2022] POST /identity/v3/users => generated 301 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:22.154370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.165077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fcd8f-0fb9-4e5a-9e81-2064249b27ba admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.169786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.170397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.170778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.171056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.171721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.173448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.183474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1299/2562] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:22.187383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.198672 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf299b-cf87-458e-9cbe-0f03be897c71 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.203257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cafe8-5d4e-4739-9b54-c36a47a51c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.203908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cafe8-5d4e-4739-9b54-c36a47a51c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.204203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cafe8-5d4e-4739-9b54-c36a47a51c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.204474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cafe8-5d4e-4739-9b54-c36a47a51c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.205138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cafe8-5d4e-4739-9b54-c36a47a51c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.206742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cafe8-5d4e-4739-9b54-c36a47a51c63 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.218293 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1264/2563] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:22.222511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.232927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08693ac-9c2b-4745-90e8-c40edda60810 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.237499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.243824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.243824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.243824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] PATH_INFO: `/v3/projects/2d09ef68b9d44087b8ec195a516935f6/users/8c8cebccba324abba1bd5909f60685de/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.253427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d09ef68b9d44087b8ec195a516935f6, user_id=8c8cebccba324abba1bd5909f60685de, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.256086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.337894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1300/2564] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:22 2022] PUT /identity/v3/projects/2d09ef68b9d44087b8ec195a516935f6/users/8c8cebccba324abba1bd5909f60685de/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:22.348065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.348348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.348647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.376693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] MFA Rules not processed for user `8c8cebccba324abba1bd5909f60685de`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:22.377970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:22.446129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1265/2565] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:22 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:22.451142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.530457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928a579-e821-480a-b25c-1e8612b16031 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.536247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.537664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.537664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.537664 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.538366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.540317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.724346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1301/2566] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:22 2022] POST /identity/v3/projects => generated 403 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:22.729332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.741465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cbeec-1b2f-4a63-8ca1-a444f1287eff None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.777670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:22.804083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.804458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.805049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.805165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.805762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.807681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.909619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1266/2567] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:22 2022] POST /identity/v3/users => generated 308 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:22.921600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.933101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129dbb69-1754-4437-b34a-50cf46d0b1c0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.938069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.938702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.938979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.939255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.939893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.941596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:22.956910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1302/2568] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:22.959980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:22.971462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d701f-6293-4994-9ac6-45064cf3e2d7 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:22.976306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:22.976947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:22.977213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:22.977623 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] PATH_INFO: `/v3/projects/2a46cb65ee4044b0877e46472d526214/users/eed7661f7f35452e8178b38386fd9fd6/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:22.993738 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a46cb65ee4044b0877e46472d526214, user_id=eed7661f7f35452e8178b38386fd9fd6, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:22.996740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:23.082808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1267/2569] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:22 2022] PUT /identity/v3/projects/2a46cb65ee4044b0877e46472d526214/users/eed7661f7f35452e8178b38386fd9fd6/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:23.087866 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:23.104767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc86c7-e737-40ea-929d-71fedf82430f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:23.173586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58297bb-61aa-40a3-9dcd-cabe676783a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:23.174293 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58297bb-61aa-40a3-9dcd-cabe676783a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:23.174591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58297bb-61aa-40a3-9dcd-cabe676783a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:23.174852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58297bb-61aa-40a3-9dcd-cabe676783a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:23.175491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58297bb-61aa-40a3-9dcd-cabe676783a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:23.177150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58297bb-61aa-40a3-9dcd-cabe676783a6 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:23.188124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1303/2570] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:23.192465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:23.203582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1339d-6df6-4e73-90af-78399c0819a0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:23.213865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:23.213865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:23.213865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:23.213865 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] PATH_INFO: `/v3/projects/2a46cb65ee4044b0877e46472d526214/users/eed7661f7f35452e8178b38386fd9fd6/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:23.214547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a46cb65ee4044b0877e46472d526214, user_id=eed7661f7f35452e8178b38386fd9fd6, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:23.214547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:23.225322 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1268/2571] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:23 2022] PUT /identity/v3/projects/2a46cb65ee4044b0877e46472d526214/users/eed7661f7f35452e8178b38386fd9fd6/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:23.237184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:23.237486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:23.237772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:23.274613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] MFA Rules not processed for user `eed7661f7f35452e8178b38386fd9fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:23.284631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:23.295970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:23.366830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1304/2572] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:23.566627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:23.568320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:23.634555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3fc9e-4a01-402f-81c1-f58e9cafe074 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:23.643727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514bb7f1-7c0b-4d95-b7e2-a3bd5c8e922d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:23.644582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bb7f1-7c0b-4d95-b7e2-a3bd5c8e922d service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:23.645024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bb7f1-7c0b-4d95-b7e2-a3bd5c8e922d service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:23.645300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514bb7f1-7c0b-4d95-b7e2-a3bd5c8e922d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:23.655541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bb7f1-7c0b-4d95-b7e2-a3bd5c8e922d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:23.657047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514bb7f1-7c0b-4d95-b7e2-a3bd5c8e922d service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:23.666999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1269/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:23 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:23.859958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:23.922123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d6b84-7ab6-45ab-8b46-a4a9625fe969 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:23.926859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:23.927491 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:23.927771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:23.928047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] PATH_INFO: `/v3/users/8c8cebccba324abba1bd5909f60685de` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:23.929195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8cebccba324abba1bd5909f60685de)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:23.931629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:24.220568 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8cebccba324abba1bd5909f60685de'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.221242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8cebccba324abba1bd5909f60685de'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.229507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8cebccba324abba1bd5909f60685de'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.230152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8cebccba324abba1bd5909f60685de'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.349017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8cebccba324abba1bd5909f60685de'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.367816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1305/2574] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:23 2022] DELETE /identity/v3/users/8c8cebccba324abba1bd5909f60685de => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:24.372380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:24.372906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:24.373028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:24.391826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:24.393064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:24.450124 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1270/2575] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:24.460358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:24.462354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:24.495246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:24.520419 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3f7c-3f70-470e-a309-8eb512819d7b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:24.524783 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:24.525475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:24.525755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:24.526030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:24.537127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:24.540511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:24.550460 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1306/2576] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:24.725925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:24.727888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:24.740160 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb77564-6f37-42fb-a954-167b82967f5b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:24.745253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:24.745882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:24.746161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:24.746435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] PATH_INFO: `/v3/users/61fed0266f34458fb2fde04bacd01d67` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:24.747552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61fed0266f34458fb2fde04bacd01d67)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:24.749488 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:24.864588 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:24.987353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61fed0266f34458fb2fde04bacd01d67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.988026 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61fed0266f34458fb2fde04bacd01d67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.994928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61fed0266f34458fb2fde04bacd01d67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:24.995573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61fed0266f34458fb2fde04bacd01d67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.071262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:25.072806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:25.089381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449a3fd-dbe4-451a-883f-8cbb9e7ff655 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:25.094723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:25.095379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:25.098229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:25.098229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] PATH_INFO: `/v3/projects/2d09ef68b9d44087b8ec195a516935f6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:25.098229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d09ef68b9d44087b8ec195a516935f6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:25.099021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:25.120923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-780c9c71-174f-4e2a-9bbe-2d5ecdb7af51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61fed0266f34458fb2fde04bacd01d67'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.140100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1271/2577] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:24 2022] DELETE /identity/v3/users/61fed0266f34458fb2fde04bacd01d67 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:25.144265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:25.144548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:25.144853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:25.163465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:25.164696 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:25.226080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1272/2578] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:25.235874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:25.242990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:25.282722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d09ef68b9d44087b8ec195a516935f6', 'request_id': 'req-ba943ed9-a926-4504-a692-fb98fdcc6e23'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.289043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d09ef68b9d44087b8ec195a516935f6', 'request_id': 'req-ba943ed9-a926-4504-a692-fb98fdcc6e23'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.295537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a88adb-021e-4418-907b-0ac4319b6077 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:25.344308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:25.344928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:25.345259 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:25.345535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:25.357577 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:25.394336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d09ef68b9d44087b8ec195a516935f6', 'request_id': 'req-ba943ed9-a926-4504-a692-fb98fdcc6e23'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.405367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:25.406249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] The token cache is being invalidate because project 2d09ef68b9d44087b8ec195a516935f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:25.406542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.406806 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:25.407217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.408690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1307/2579] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:25 2022] DELETE /identity/v3/projects/2d09ef68b9d44087b8ec195a516935f6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:25.412627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:25.414071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:25.418096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1273/2580] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:42:25.431694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba943ed9-a926-4504-a692-fb98fdcc6e23 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:25.436727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:25.437435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:25.438739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:25.438739 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] PATH_INFO: `/v3/users/eed7661f7f35452e8178b38386fd9fd6` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:25.439107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed7661f7f35452e8178b38386fd9fd6)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:25.441066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:25.677835 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed7661f7f35452e8178b38386fd9fd6'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.678583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed7661f7f35452e8178b38386fd9fd6'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.686441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed7661f7f35452e8178b38386fd9fd6'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.687175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed7661f7f35452e8178b38386fd9fd6'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.806787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed7661f7f35452e8178b38386fd9fd6'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:25.825035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1308/2581] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:25 2022] DELETE /identity/v3/users/eed7661f7f35452e8178b38386fd9fd6 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:26.036441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:26.045815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:26.053239 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df958e73-7897-42a0-851f-3312eca34da8 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:26.120636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:26.121299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:26.121574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:26.121843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] PATH_INFO: `/v3/projects/8b0483abbf6b4988951239bc78ab5236` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:26.122940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0483abbf6b4988951239bc78ab5236)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:26.124882 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:26.307189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0483abbf6b4988951239bc78ab5236', 'request_id': 'req-700725da-0e46-49ab-b68f-f8222f64bb7a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.314546 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0483abbf6b4988951239bc78ab5236', 'request_id': 'req-700725da-0e46-49ab-b68f-f8222f64bb7a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.324485 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0483abbf6b4988951239bc78ab5236', 'request_id': 'req-700725da-0e46-49ab-b68f-f8222f64bb7a'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.417904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] The token cache is being invalidate because project 8b0483abbf6b4988951239bc78ab5236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:26.418678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.418795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.420281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1274/2582] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:26 2022] DELETE /identity/v3/projects/8b0483abbf6b4988951239bc78ab5236 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:26.424525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:26.426048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:26.439630 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28028d4b-bec7-4acc-88d3-a1f49ffe7aca admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:26.465362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:26.496573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:26.497373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:26.497723 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:26.498049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] PATH_INFO: `/v3/users/1c03f8cce9e3491b8bfa21e1b256ffc5` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:26.499203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c03f8cce9e3491b8bfa21e1b256ffc5)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:26.501403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:26.524644 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:26.526745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:26.541524 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700725da-0e46-49ab-b68f-f8222f64bb7a admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:26.546389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:26.547149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:26.547425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:26.547701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] PATH_INFO: `/v3/projects/2a46cb65ee4044b0877e46472d526214` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:26.548884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a46cb65ee4044b0877e46472d526214)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:26.551308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:26.828722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:26.839943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c03f8cce9e3491b8bfa21e1b256ffc5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.840490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a46cb65ee4044b0877e46472d526214', 'request_id': 'req-44b03d72-253e-44b3-a93b-44baaf0fa828'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.840730 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c03f8cce9e3491b8bfa21e1b256ffc5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.847105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a46cb65ee4044b0877e46472d526214', 'request_id': 'req-44b03d72-253e-44b3-a93b-44baaf0fa828'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.848686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c03f8cce9e3491b8bfa21e1b256ffc5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.849438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c03f8cce9e3491b8bfa21e1b256ffc5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.857308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a46cb65ee4044b0877e46472d526214', 'request_id': 'req-44b03d72-253e-44b3-a93b-44baaf0fa828'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:26.983039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-931c23a9-2fdb-4821-9fc7-c60dd7855ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c03f8cce9e3491b8bfa21e1b256ffc5'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.001014 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1309/2583] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:26 2022] DELETE /identity/v3/users/1c03f8cce9e3491b8bfa21e1b256ffc5 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:27.047130 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] The token cache is being invalidate because project 2a46cb65ee4044b0877e46472d526214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:27.047489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.055860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-44b03d72-253e-44b3-a93b-44baaf0fa828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.055860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1275/2584] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:26 2022] DELETE /identity/v3/projects/2a46cb65ee4044b0877e46472d526214 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:27.056306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.056306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.056306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.075691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:27.076851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.136537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1310/2585] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:27.145404 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.145716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.145992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.167034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:27.168312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.176411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1276/2586] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:27.181324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.190834 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af3c5e3-f300-4637-85ac-11faf20c2f86 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.196237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.196840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.197152 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.197429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.198094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:27.198385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.200004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.215776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1311/2587] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:27.230096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.239538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a42dcb-3db7-4668-8166-64386d2ddb26 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.244296 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.245095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.245365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.245633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.246253 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.248146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.328516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1277/2588] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:27 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:27.332737 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.343696 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26165e0-d177-4213-b198-ee443922bc8c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.361389 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.363093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.401674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.402280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.402557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.402830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.403490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.405439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.424219 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:27.430511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e38826-5beb-421a-ab25-5a7fac90ff49 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.435362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.436003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.436283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.436561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] PATH_INFO: `/v3/users/94a7b98e02ab4d699248258e44e21856` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.437744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a7b98e02ab4d699248258e44e21856)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.439711 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.486251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1312/2589] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:27 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:27.487083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.488784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.501117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed7447-959c-4f75-8f08-e7eaa88362d3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.505803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.506415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.506695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.506970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] PATH_INFO: `/v3/projects/7b5e4ffc498848888f04ae213fde94d2` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.509021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5e4ffc498848888f04ae213fde94d2)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.510946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.655062 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a7b98e02ab4d699248258e44e21856'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.655698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a7b98e02ab4d699248258e44e21856'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.658959 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5e4ffc498848888f04ae213fde94d2', 'request_id': 'req-568caeb7-a370-4c8d-9e5b-cf65d82dc075'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.663126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a7b98e02ab4d699248258e44e21856'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.663775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a7b98e02ab4d699248258e44e21856'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.669610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5e4ffc498848888f04ae213fde94d2', 'request_id': 'req-568caeb7-a370-4c8d-9e5b-cf65d82dc075'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.749297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a7b98e02ab4d699248258e44e21856'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.767877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1278/2590] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:27 2022] DELETE /identity/v3/users/94a7b98e02ab4d699248258e44e21856 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:27.769388 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.770845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.784497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:27.828841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5e4ffc498848888f04ae213fde94d2', 'request_id': 'req-568caeb7-a370-4c8d-9e5b-cf65d82dc075'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.836722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da40af-7c60-4d02-ac5f-bcbb3c2cca9d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.841907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.842429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] The token cache is being invalidate because project 7b5e4ffc498848888f04ae213fde94d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:27.842544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.842732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.842860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.843048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.843403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-568caeb7-a370-4c8d-9e5b-cf65d82dc075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:27.843815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.845241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1313/2591] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:27 2022] DELETE /identity/v3/projects/7b5e4ffc498848888f04ae213fde94d2 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:27.845423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.846788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.847295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.848188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.856908 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1279/2592] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:27.857888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.859768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.874936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff7c0a-ce44-49b4-abd0-589803456ee1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.875117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:27.875534 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.877677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.878281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.878556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.878826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] PATH_INFO: `/v3/users/d7700f5197c24fb0afea938b7dbe6041` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.879969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7700f5197c24fb0afea938b7dbe6041)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.881899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.884441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1314/2593] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:27.885261 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.886660 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.898173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f040210-593f-4418-bdbb-80e08b5be0f1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.903864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.904889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.904889 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.905111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.905683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.907337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:27.918047 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1315/2594] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:27.919009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:27.920628 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:27.977909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50467e39-3b94-43b1-ba64-bd1f485162e8 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:27.981667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:27.982268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:27.982548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:27.982822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:27.991848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:27.994375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.004226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1316/2595] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:28.005108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:28.015200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39cd74a-278f-41b9-93b9-f4b102bbb149 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:28.020005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87507f35-122d-4423-877e-239699bff5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:28.020644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f35-122d-4423-877e-239699bff5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.021019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f35-122d-4423-877e-239699bff5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.021292 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87507f35-122d-4423-877e-239699bff5aa admin admin] PATH_INFO: `/v3/projects/d6de93a3566142719a5496378c84ec34/users/0d191a11ca124877adfce3d2caef4c8d/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.036573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87507f35-122d-4423-877e-239699bff5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6de93a3566142719a5496378c84ec34, user_id=0d191a11ca124877adfce3d2caef4c8d, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:28.041404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87507f35-122d-4423-877e-239699bff5aa admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.072061 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7700f5197c24fb0afea938b7dbe6041'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.072927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7700f5197c24fb0afea938b7dbe6041'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.080114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7700f5197c24fb0afea938b7dbe6041'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.080807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7700f5197c24fb0afea938b7dbe6041'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.161539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1317/2596] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:28 2022] PUT /identity/v3/projects/d6de93a3566142719a5496378c84ec34/users/0d191a11ca124877adfce3d2caef4c8d/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:28.172656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.172974 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.173280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.184101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7700f5197c24fb0afea938b7dbe6041'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.187949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:28.203910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] MFA Rules not processed for user `0d191a11ca124877adfce3d2caef4c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:28.205016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1280/2597] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:27 2022] DELETE /identity/v3/users/d7700f5197c24fb0afea938b7dbe6041 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:28.205626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:28.290556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1318/2598] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:28.294775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:28.296286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:28.364559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1f135-12f3-4627-a37d-aae3d6f5ca09 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:28.369410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:28.370052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.370335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.370609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.371257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:28.373235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.542770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:28.544307 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:28.558657 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028cefc-49cf-472a-a925-b3aea0912d40 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:28.568578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:28.568578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.568578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.568578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] PATH_INFO: `/v3/projects/bd90fe960dd44e569d79910393cd5556` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.569067 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd90fe960dd44e569d79910393cd5556)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:28.569146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1281/2599] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:28 2022] POST /identity/v3/projects => generated 367 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:28.570718 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.574142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:28.583961 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:28.639195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db0915-7794-46c3-b90a-7c06d6e37c75 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:28.643730 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:28.644327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.644628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.649896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.649896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:28.649896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.773056 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd90fe960dd44e569d79910393cd5556', 'request_id': 'req-9feeae32-7107-47aa-b803-4dba2e16a628'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.781733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd90fe960dd44e569d79910393cd5556', 'request_id': 'req-9feeae32-7107-47aa-b803-4dba2e16a628'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.786459 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1282/2600] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:28 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:28.787339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:28.790507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:28.805763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f79e9-3430-4e9a-833e-2edf705cc024 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:28.862990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:28.863627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.863985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.864189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] PATH_INFO: `/v3/projects/215c1e6668d54327be09ef84c2d48b02` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.865355 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=215c1e6668d54327be09ef84c2d48b02)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:28.867405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.937836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd90fe960dd44e569d79910393cd5556', 'request_id': 'req-9feeae32-7107-47aa-b803-4dba2e16a628'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.949140 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] The token cache is being invalidate because project bd90fe960dd44e569d79910393cd5556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:28.949629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.950128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:28.951600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1319/2601] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:28 2022] DELETE /identity/v3/projects/bd90fe960dd44e569d79910393cd5556 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:28.952367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:28.953755 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:28.966463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feeae32-7107-47aa-b803-4dba2e16a628 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:28.971020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28151c-382c-4d92-bee8-b4f7639c6630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:28.971594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28151c-382c-4d92-bee8-b4f7639c6630 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.971873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28151c-382c-4d92-bee8-b4f7639c6630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.972150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28151c-382c-4d92-bee8-b4f7639c6630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.972779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28151c-382c-4d92-bee8-b4f7639c6630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:28.974395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28151c-382c-4d92-bee8-b4f7639c6630 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:28.985343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1320/2602] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:28.987328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:28.991727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:28.991727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:28.994987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '215c1e6668d54327be09ef84c2d48b02', 'request_id': 'req-a04d477a-de7c-4c2f-aab6-07a062c4a435'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:29.001640 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '215c1e6668d54327be09ef84c2d48b02', 'request_id': 'req-a04d477a-de7c-4c2f-aab6-07a062c4a435'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:29.007017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '215c1e6668d54327be09ef84c2d48b02', 'request_id': 'req-a04d477a-de7c-4c2f-aab6-07a062c4a435'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:29.008170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:29.009508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.067563 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1321/2603] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:29.068546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.080574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999b101-9f3e-4ee3-b85e-830e0d1d6607 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.085422 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.086506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.086790 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.087094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] PATH_INFO: `/v3/projects/73b2f09b32c74bb1b5f894f586f5dccc/users/c5bb2961ed56410da2987c195042a353/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.101424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b2f09b32c74bb1b5f894f586f5dccc, user_id=c5bb2961ed56410da2987c195042a353, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.103988 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] The token cache is being invalidate because project 215c1e6668d54327be09ef84c2d48b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:29.104163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.104257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:29.104940 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a04d477a-de7c-4c2f-aab6-07a062c4a435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:29.106422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1283/2604] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:28 2022] DELETE /identity/v3/projects/215c1e6668d54327be09ef84c2d48b02 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:29.107796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba4f6e-93d2-428a-983d-5af870f7e9a4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.108074 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba4f6e-93d2-428a-983d-5af870f7e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.108346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba4f6e-93d2-428a-983d-5af870f7e9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.109684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:29.127962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6bba4f6e-93d2-428a-983d-5af870f7e9a4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:29.129152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6bba4f6e-93d2-428a-983d-5af870f7e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.137688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1284/2605] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:29.138815 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.139091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.144656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.157693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:29.158859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.166785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1285/2606] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:29.167528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.176529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307312da-9df4-4ef2-8b16-6d028a534627 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.182003 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.182573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.182849 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.183116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.183741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:29.184036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.185642 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abc1b4-6758-4a85-b021-f4dd8c949344 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.197755 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1286/2607] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:29.198903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.199183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.199453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.218126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:29.219312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.223666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1322/2608] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:29 2022] PUT /identity/v3/projects/73b2f09b32c74bb1b5f894f586f5dccc/users/c5bb2961ed56410da2987c195042a353/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:29.229967 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.281790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1287/2609] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:29.282710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.284104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.288068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64462f1c-9f9b-44bd-adee-20f7cc505b5a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.293382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.294074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.294353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.294627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.295260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.295889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a904e8a-b100-4f81-bffb-f5f8c34c21a2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.297338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.300536 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.301189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.301515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.301790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.305025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.305025 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.314505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1288/2610] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:29.315409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.325807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef23e0-64e1-4d73-beeb-b25986c00e3d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.330707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.331320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.331601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.331856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.332478 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:29.332776 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.334461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.346740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1289/2611] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:29.347552 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.356895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e58a0-db50-49da-8041-0b305b94ff72 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.361572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.362196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.362472 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.362766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] PATH_INFO: `/v3/projects/73b2f09b32c74bb1b5f894f586f5dccc/users/c5bb2961ed56410da2987c195042a353/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.364661 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73b2f09b32c74bb1b5f894f586f5dccc, user_id=c5bb2961ed56410da2987c195042a353, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.368303 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.378604 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1323/2612] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:29.379486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.381903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1290/2613] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:29 2022] PUT /identity/v3/projects/73b2f09b32c74bb1b5f894f586f5dccc/users/c5bb2961ed56410da2987c195042a353/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:29.386530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.430704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:29.443343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816eb6-e48e-4aaa-8bde-2a6996335c74 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.448252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3098a5-7f01-4bd1-b46c-7aa8e3ee993b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.448843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3098a5-7f01-4bd1-b46c-7aa8e3ee993b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.449146 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3098a5-7f01-4bd1-b46c-7aa8e3ee993b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.449417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3098a5-7f01-4bd1-b46c-7aa8e3ee993b admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.450038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3098a5-7f01-4bd1-b46c-7aa8e3ee993b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.451929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3098a5-7f01-4bd1-b46c-7aa8e3ee993b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.452926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a040c9-8b3b-471b-811d-230e9571a075 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.457634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.458227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.458506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.458779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.459428 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.461914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.584520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1324/2614] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:29.585812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.586326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.586576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.614546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] MFA Rules not processed for user `c5bb2961ed56410da2987c195042a353`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:29.615775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.620636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1291/2615] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:29.624637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.633498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42995b-df1a-4624-92a7-4af3a264d913 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.684528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1325/2616] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:29.685582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.691949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.693051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.693453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.693798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.694590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.696966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.721727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:29.751176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce47b1-14a9-43c0-a660-af833541b653 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.755896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.756504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.756785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.757085 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.757899 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.759508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.769063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1326/2617] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:29.770071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.771645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:29.817985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1292/2618] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:29 2022] POST /identity/v3/users => generated 284 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:29.818935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.831208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b956e7-fbae-48c3-a339-2bfa330fc4cc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.831535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed7dc7-2ce3-41a5-9ab9-a63b08370e14 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.835242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.836655 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.836958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.837395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.838676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.839283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.839560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.839832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.840525 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.842152 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.846462 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.847878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.853208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1293/2619] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:29.854089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.861554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1327/2620] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:29.867846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.867969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d687b-bf4f-4bda-9b3b-113a05786eb8 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.869722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.870447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.870819 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.871185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.871963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.872736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af92f9-9239-46ed-b88b-e28cc616f3e8 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.873889 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.877619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb434bc-e58f-4c04-9e55-10a1296231f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.878268 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb434bc-e58f-4c04-9e55-10a1296231f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.878550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb434bc-e58f-4c04-9e55-10a1296231f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.878826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb434bc-e58f-4c04-9e55-10a1296231f4 admin admin] PATH_INFO: `/v3/projects/7a81589530be4b5ea7d1e08c752a0ca8/users/2d0c0d33e4754125abf3b5cae6f0dfe6/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.886649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1294/2621] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:29.890633 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.896874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb434bc-e58f-4c04-9e55-10a1296231f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a81589530be4b5ea7d1e08c752a0ca8, user_id=2d0c0d33e4754125abf3b5cae6f0dfe6, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.900671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb434bc-e58f-4c04-9e55-10a1296231f4 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.900962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f94224-7752-4c1d-b532-5e5cc6f763e9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.905917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.906504 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.906781 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.907048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.907676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.909270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:29.916673 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:29.921393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1295/2622] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:29.925494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:29.934762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356086a-642d-41af-b241-b208da81eca6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:29.939441 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a1a35-4b5e-4870-b5f8-8392626c313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:29.940056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a1a35-4b5e-4870-b5f8-8392626c313a admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:29.940339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a1a35-4b5e-4870-b5f8-8392626c313a admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:29.940612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a1a35-4b5e-4870-b5f8-8392626c313a admin admin] PATH_INFO: `/v3/projects/377e2eda387845078df49f56396654d6/users/389ed248f4974d3dbca8ef00086782ad/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:29.955369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a1a35-4b5e-4870-b5f8-8392626c313a admin admin] RBAC: Authorizing `identity:create_grant(project_id=377e2eda387845078df49f56396654d6, user_id=389ed248f4974d3dbca8ef00086782ad, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:29.958027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a1a35-4b5e-4870-b5f8-8392626c313a admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.041164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1296/2623] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:29 2022] PUT /identity/v3/projects/377e2eda387845078df49f56396654d6/users/389ed248f4974d3dbca8ef00086782ad/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:42:30.050141 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1328/2624] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:29 2022] PUT /identity/v3/projects/7a81589530be4b5ea7d1e08c752a0ca8/users/2d0c0d33e4754125abf3b5cae6f0dfe6/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:30.051534 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.051968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.052242 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.053386 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.053669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.053943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.081968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] MFA Rules not processed for user `389ed248f4974d3dbca8ef00086782ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:30.082217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] MFA Rules not processed for user `2d0c0d33e4754125abf3b5cae6f0dfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:30.083213 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:30.083358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:30.131725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:30.156116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1329/2625] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:30.160114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1297/2626] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:30.160519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.172604 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.174012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:30.223848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40b3b7-7cbd-4af9-a8ac-d2468fbfed88 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.228731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.229370 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.229651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.229926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.230559 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.232260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664113e-5a85-4042-b9d3-b5e38c02edb0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.232482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.235906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.236487 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.236765 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.237067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.246575 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.249060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.260056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1298/2627] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:42:30.345545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1330/2628] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:30 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:30.349554 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.361270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd604c-ca25-40ff-b7e7-c85dc622cec9 service glance] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.374898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:30.421699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.422425 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.422708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.423165 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.423805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.426612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.499330 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1299/2629] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:30 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:30.503415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.514468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043a6dd-e92f-46ff-b86e-1dd8887fbfc1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.519564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.520170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.520510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.520794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.521468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.523151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.539501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1331/2630] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:30.542319 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.555980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf1f7-24e4-4ad1-84e1-19c730cdf1c9 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.562564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.563228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.563514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.563790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] PATH_INFO: `/v3/projects/a4c1adf9d8604cfa88ee53650fbe9f65/users/fb1a7698114246e08fcc65e06ba6048d/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.579212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c1adf9d8604cfa88ee53650fbe9f65, user_id=fb1a7698114246e08fcc65e06ba6048d, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.582167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.720721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1300/2631] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:30 2022] PUT /identity/v3/projects/a4c1adf9d8604cfa88ee53650fbe9f65/users/fb1a7698114246e08fcc65e06ba6048d/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:30.724813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.735488 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87083d0-998d-4179-a513-64414889d6f2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.791461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1aacd8-4862-4faf-b15d-a514064863fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.792053 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aacd8-4862-4faf-b15d-a514064863fe admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.792328 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aacd8-4862-4faf-b15d-a514064863fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.792636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aacd8-4862-4faf-b15d-a514064863fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.793305 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1aacd8-4862-4faf-b15d-a514064863fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.794880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1aacd8-4862-4faf-b15d-a514064863fe admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.804471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1332/2632] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:30.808958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.819603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb77a8-f826-46ef-ad4e-ddfaa7d9551f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:30.824561 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:30.825251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.825531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.825800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] PATH_INFO: `/v3/projects/a4c1adf9d8604cfa88ee53650fbe9f65/users/fb1a7698114246e08fcc65e06ba6048d/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.827859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4c1adf9d8604cfa88ee53650fbe9f65, user_id=fb1a7698114246e08fcc65e06ba6048d, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:30.833829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:30.844343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1301/2633] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:30 2022] PUT /identity/v3/projects/a4c1adf9d8604cfa88ee53650fbe9f65/users/fb1a7698114246e08fcc65e06ba6048d/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:30.856188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:30.856472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:30.856806 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:30.885725 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] MFA Rules not processed for user `fb1a7698114246e08fcc65e06ba6048d`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:30.890435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:30.933648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:30.957709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1333/2634] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:30.973395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:30.975095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:31.031892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b640b0-9641-4289-94af-573b56fa25c3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:31.035490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:31.036080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:31.036362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:31.036627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:31.045836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:31.047257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:31.057415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1302/2635] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:31.341441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:31.350808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db1cd0-33ab-4730-9552-31bf221b3d2c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:31.354878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283245bc-fa7e-4308-b31f-619c1ee62bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:31.355467 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283245bc-fa7e-4308-b31f-619c1ee62bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:31.355743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283245bc-fa7e-4308-b31f-619c1ee62bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:31.356009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283245bc-fa7e-4308-b31f-619c1ee62bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:31.415346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283245bc-fa7e-4308-b31f-619c1ee62bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:31.416887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283245bc-fa7e-4308-b31f-619c1ee62bf3 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:31.426564 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1334/2636] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:33.517562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:33.586864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da22ffb-d12c-47e0-8ba2-0593fb31ff22 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:33.591509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:33.592136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:33.592411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:33.592680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] PATH_INFO: `/v3/users/0d191a11ca124877adfce3d2caef4c8d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:33.593824 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d191a11ca124877adfce3d2caef4c8d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:33.595718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:33.612593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:33.831051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d191a11ca124877adfce3d2caef4c8d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:33.831843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d191a11ca124877adfce3d2caef4c8d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:33.840766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d191a11ca124877adfce3d2caef4c8d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:33.844305 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d191a11ca124877adfce3d2caef4c8d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:33.971587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d191a11ca124877adfce3d2caef4c8d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:33.991291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1303/2637] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:33 2022] DELETE /identity/v3/users/0d191a11ca124877adfce3d2caef4c8d => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:33.995988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:33.996475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:33.996578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:34.016196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:34.017358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:34.083144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1335/2638] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:34.094832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:34.096338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:34.154393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6fd2c-7e23-4559-8ea7-e0dfe667a649 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:34.158992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:34.159628 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:34.159904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:34.160187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:34.170720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:34.172719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:34.183391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1304/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:34.609262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:34.610851 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:34.627341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04e5c5-b107-4c5a-b73f-1695ad3453ed None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:34.630397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:34.631052 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:34.631329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:34.631602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] PATH_INFO: `/v3/projects/d6de93a3566142719a5496378c84ec34` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:34.633342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6de93a3566142719a5496378c84ec34)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:34.635241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:34.791302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6de93a3566142719a5496378c84ec34', 'request_id': 'req-a0b3cb69-80be-433d-8101-917aff234a68'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:34.798337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6de93a3566142719a5496378c84ec34', 'request_id': 'req-a0b3cb69-80be-433d-8101-917aff234a68'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:34.869129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6de93a3566142719a5496378c84ec34', 'request_id': 'req-a0b3cb69-80be-433d-8101-917aff234a68'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:34.884497 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] The token cache is being invalidate because project d6de93a3566142719a5496378c84ec34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:34.885142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:34.886420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:34.888050 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1336/2640] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:34 2022] DELETE /identity/v3/projects/d6de93a3566142719a5496378c84ec34 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:34.892149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:34.907797 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09624d-6ab5-4712-82fb-bdc386b42112 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:34.973817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:34.975097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:34.975386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:34.975667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] PATH_INFO: `/v3/users/c5bb2961ed56410da2987c195042a353` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:34.976832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5bb2961ed56410da2987c195042a353)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:34.978861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:35.167157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:35.176111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5bb2961ed56410da2987c195042a353'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.177188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5bb2961ed56410da2987c195042a353'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.185426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5bb2961ed56410da2987c195042a353'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.186087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5bb2961ed56410da2987c195042a353'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.269675 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-820cf0a9-ee75-4a5a-838b-0e7318b37257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5bb2961ed56410da2987c195042a353'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.288302 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1305/2641] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:34 2022] DELETE /identity/v3/users/c5bb2961ed56410da2987c195042a353 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:35.688104 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:35.689688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:35.700336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:35.706850 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3cb69-80be-433d-8101-917aff234a68 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:35.771499 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:35.772191 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:35.772453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:35.772728 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] PATH_INFO: `/v3/projects/73b2f09b32c74bb1b5f894f586f5dccc` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:35.775114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73b2f09b32c74bb1b5f894f586f5dccc)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:35.777122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:35.864266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73b2f09b32c74bb1b5f894f586f5dccc', 'request_id': 'req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.871060 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73b2f09b32c74bb1b5f894f586f5dccc', 'request_id': 'req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.943100 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73b2f09b32c74bb1b5f894f586f5dccc', 'request_id': 'req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.954901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] The token cache is being invalidate because project 73b2f09b32c74bb1b5f894f586f5dccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:35.955199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.957124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8d9c6d7b-89dd-4f42-8d83-0ea3f758b566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:35.957469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1337/2642] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:35 2022] DELETE /identity/v3/projects/73b2f09b32c74bb1b5f894f586f5dccc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:35.962846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:35.963557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:35.963792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:35.985094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:35.986300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:36.047332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1306/2643] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:42:36.055658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.055940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.056262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.076770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:36.078013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:36.086330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1338/2644] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:36.090655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.100251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462706fb-b5b8-4142-8c32-55d1bd4e9f9b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.105244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.105855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.106116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.106387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.107027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:36.107329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.108958 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:36.121300 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1307/2645] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:36.125674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.136094 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6221db-fe9c-4b01-89f5-8382d4e8187d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.141008 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.141635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.141949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.142226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.142909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.144939 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:36.266123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1339/2646] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:36 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:36.270745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.281508 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e6248-fd53-48dc-bc23-577c3319f6c6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.339933 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.340540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.340823 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.341128 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.341768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.343676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:36.436318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1308/2647] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:36 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:36.440359 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.450849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec00be6-6df8-4d20-9737-f0a4e5733634 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.455501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.456201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.456482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.456759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.457430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.459029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:36.472840 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1340/2648] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:36.476752 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.497496 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee6ff3-1510-4ebd-9553-89f2c99c538f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.502851 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57c030-2d99-4438-b813-47bd45b1385d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.503493 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57c030-2d99-4438-b813-47bd45b1385d admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.503775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57c030-2d99-4438-b813-47bd45b1385d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.504049 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57c030-2d99-4438-b813-47bd45b1385d admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.504959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57c030-2d99-4438-b813-47bd45b1385d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.506569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57c030-2d99-4438-b813-47bd45b1385d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:36.517649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1309/2649] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:36.522543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.534090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b2215-2851-4f80-861f-540cec09e471 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.541378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.541378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.541378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.541378 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] PATH_INFO: `/v3/projects/a0a1778b4d914492ab96929aa2e0984e/users/ca0e8a9fc65b4c34b7c7eb2d81bb9136/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.564373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a1778b4d914492ab96929aa2e0984e, user_id=ca0e8a9fc65b4c34b7c7eb2d81bb9136, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.567282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:36.671327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1341/2650] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:36 2022] PUT /identity/v3/projects/a0a1778b4d914492ab96929aa2e0984e/users/ca0e8a9fc65b4c34b7c7eb2d81bb9136/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:36.686935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.687223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.687497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.737464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] MFA Rules not processed for user `ca0e8a9fc65b4c34b7c7eb2d81bb9136`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:36.738690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:36.748060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:36.822826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1310/2651] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:36.828671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:36.861544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:36.892852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a56de7-4b1b-490f-bf5c-38b4917e1c2a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:36.897603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:36.899302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:36.899582 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:36.899861 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:36.900492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:36.902442 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.038156 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1342/2652] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:36 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:37.042261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:37.053117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc59e69-c4b9-4b8e-b4cc-ae7a3b4e9e63 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:37.111624 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:37.112143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.112419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.112697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.113440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:37.115344 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.196121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1311/2653] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:37 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:37.200632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:37.213665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8e8cc-2361-475f-aa42-20c33c8eff0d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:37.218405 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:37.219020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.219300 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.219575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.220214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:37.221867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.232542 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1343/2654] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:37.236745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:37.247238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f0c03-e0bc-4a00-a3ed-8f17af3ec866 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:37.253660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:37.254365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.254648 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.254925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] PATH_INFO: `/v3/projects/c69878eaff174ad9885ce990611d2881/users/23c655cc103b4d38a6d9cce43bee0ef2/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.271201 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69878eaff174ad9885ce990611d2881, user_id=23c655cc103b4d38a6d9cce43bee0ef2, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:37.275101 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.379699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1312/2655] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:37 2022] PUT /identity/v3/projects/c69878eaff174ad9885ce990611d2881/users/23c655cc103b4d38a6d9cce43bee0ef2/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:37.385029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:37.395203 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0898173-5b5d-49b6-9a7c-d0dfe7003bf3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:37.454726 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f5f8e-4fcb-45cd-a54a-7dba16d7ef09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:37.455343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f5f8e-4fcb-45cd-a54a-7dba16d7ef09 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.455622 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f5f8e-4fcb-45cd-a54a-7dba16d7ef09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.455895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f5f8e-4fcb-45cd-a54a-7dba16d7ef09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.456539 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f5f8e-4fcb-45cd-a54a-7dba16d7ef09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:37.458185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f5f8e-4fcb-45cd-a54a-7dba16d7ef09 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.468793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1344/2656] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:37.473468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:37.484756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdc445-1abc-4e5a-84a8-18eeaf2106a5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:37.489956 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:37.490774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.491060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.491334 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] PATH_INFO: `/v3/projects/c69878eaff174ad9885ce990611d2881/users/23c655cc103b4d38a6d9cce43bee0ef2/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.494234 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69878eaff174ad9885ce990611d2881, user_id=23c655cc103b4d38a6d9cce43bee0ef2, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:37.496848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.508151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1313/2657] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:37 2022] PUT /identity/v3/projects/c69878eaff174ad9885ce990611d2881/users/23c655cc103b4d38a6d9cce43bee0ef2/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:37.519247 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.519535 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.519814 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.549147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] MFA Rules not processed for user `23c655cc103b4d38a6d9cce43bee0ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:37.550374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:37.627699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1345/2658] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:37.639255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:37.641134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:37.702816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2b91-94be-46f6-978d-329946c6eb1d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:37.706492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:37.707093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:37.707453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:37.707657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:37.723383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:37.770440 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:37.771877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:37.780742 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1314/2659] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:44.203147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:44.204576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:44.217417 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e758b9-f5e0-45b8-ab56-2c0b51b2cfcd None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:44.220956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a414f8-0295-44ab-a346-bf644e6df79d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:44.221597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a414f8-0295-44ab-a346-bf644e6df79d service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:44.221878 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a414f8-0295-44ab-a346-bf644e6df79d service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:44.222153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a414f8-0295-44ab-a346-bf644e6df79d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:44.232103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a414f8-0295-44ab-a346-bf644e6df79d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:44.233597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a414f8-0295-44ab-a346-bf644e6df79d service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:44.243199 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1346/2660] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:47.834439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=20 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:51.225838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:51.227509 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:51.299629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b23844-566e-4dbd-acdf-db8641cfa6b1 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:51.304326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:51.305041 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:51.305317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:51.305609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] PATH_INFO: `/v3/users/2d0c0d33e4754125abf3b5cae6f0dfe6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:51.306856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0c0d33e4754125abf3b5cae6f0dfe6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:51.308876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:51.594142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0c0d33e4754125abf3b5cae6f0dfe6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:51.594795 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0c0d33e4754125abf3b5cae6f0dfe6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:51.601938 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0c0d33e4754125abf3b5cae6f0dfe6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:51.602590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0c0d33e4754125abf3b5cae6f0dfe6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:51.691216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0c0d33e4754125abf3b5cae6f0dfe6'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:51.710191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1315/2661] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:51 2022] DELETE /identity/v3/users/2d0c0d33e4754125abf3b5cae6f0dfe6 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:51.714963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:51.715457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:51.716055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:51.736235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:51.737508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:51.742441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:51.795784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1347/2662] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:51.805403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:51.806839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:51.864406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e0378-1587-4daa-ac4a-1efc992890f3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:51.867944 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:51.868537 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:51.868813 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:51.869111 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:51.878224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:51.879632 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:51.889279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1316/2663] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:52.282310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:52.283747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:52.296117 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265596-f883-4872-a2c3-56d2e7c29c6b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:52.300716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:52.301377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:52.301658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:52.302016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] PATH_INFO: `/v3/projects/7a81589530be4b5ea7d1e08c752a0ca8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:52.303074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a81589530be4b5ea7d1e08c752a0ca8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:52.305046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:52.393706 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a81589530be4b5ea7d1e08c752a0ca8', 'request_id': 'req-d48a5733-905a-4fc8-931c-1c5274f61fbc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.399666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a81589530be4b5ea7d1e08c752a0ca8', 'request_id': 'req-d48a5733-905a-4fc8-931c-1c5274f61fbc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.479829 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a81589530be4b5ea7d1e08c752a0ca8', 'request_id': 'req-d48a5733-905a-4fc8-931c-1c5274f61fbc'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.490605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] The token cache is being invalidate because project 7a81589530be4b5ea7d1e08c752a0ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:52.490907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.491516 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.493048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1348/2664] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:52 2022] DELETE /identity/v3/projects/7a81589530be4b5ea7d1e08c752a0ca8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:52.497842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:52.507156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08e2a-7e37-4bed-8f62-f77910047d56 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:52.525517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:52.562895 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:52.563513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:52.563788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:52.564056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] PATH_INFO: `/v3/users/fb1a7698114246e08fcc65e06ba6048d` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:52.565422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb1a7698114246e08fcc65e06ba6048d)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:52.567625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:52.716544 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb1a7698114246e08fcc65e06ba6048d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.717155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb1a7698114246e08fcc65e06ba6048d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.724000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb1a7698114246e08fcc65e06ba6048d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.724636 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb1a7698114246e08fcc65e06ba6048d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.788077 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-cb7e9741-67c6-43d1-a4de-7427d48e5b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb1a7698114246e08fcc65e06ba6048d'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:52.806098 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1317/2665] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:52 2022] DELETE /identity/v3/users/fb1a7698114246e08fcc65e06ba6048d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:53.230301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:53.232077 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:53.248778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a5733-905a-4fc8-931c-1c5274f61fbc admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:53.309127 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:53.310877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.311165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.311437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] PATH_INFO: `/v3/projects/a4c1adf9d8604cfa88ee53650fbe9f65` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.312584 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4c1adf9d8604cfa88ee53650fbe9f65)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:53.314566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:53.413915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:53.423187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4c1adf9d8604cfa88ee53650fbe9f65', 'request_id': 'req-97c52241-a5d6-4b86-92e8-3c1e81b8c856'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:53.429466 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4c1adf9d8604cfa88ee53650fbe9f65', 'request_id': 'req-97c52241-a5d6-4b86-92e8-3c1e81b8c856'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:53.509176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4c1adf9d8604cfa88ee53650fbe9f65', 'request_id': 'req-97c52241-a5d6-4b86-92e8-3c1e81b8c856'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:53.521142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] The token cache is being invalidate because project a4c1adf9d8604cfa88ee53650fbe9f65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:53.521437 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:53.522081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-97c52241-a5d6-4b86-92e8-3c1e81b8c856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:53.523852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1349/2666] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:53 2022] DELETE /identity/v3/projects/a4c1adf9d8604cfa88ee53650fbe9f65 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:53.528886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.529398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.529398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.553680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:53.554902 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:53.616693 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1318/2667] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:53.625519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.625807 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.626091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.649255 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:53.650557 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:53.659950 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1350/2668] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:53.664438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:53.675920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22134f8-e85b-435f-a3f6-181633fdf9b0 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:53.680743 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:53.681401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.683744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.683744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.683744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:53.683744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:53.685121 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:53.696913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1319/2669] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:53.701666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:53.710969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df78076b-16ba-4dca-90e3-478393afb324 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:53.716501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:53.717124 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.717402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.717674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.718302 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:53.720195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:53.802847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1351/2670] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:53 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:53.806861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:53.816906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d69dd-294f-474c-8d75-13a0225f130c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:53.842981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:53.873387 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:53.873979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.874258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.874528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.875169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:53.877093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:53.972645 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1320/2671] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:53 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:42:53.976776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:53.987895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7206-2e9d-4c79-bfc0-a01010eb9a97 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:53.992469 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:53.993113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:53.993394 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:53.993668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:53.994308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:53.995910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.006204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1352/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:54.010196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.020657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b72b1-04ee-47c9-ab27-cff4297053a4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.025288 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21ffcd-9148-46f9-a284-83646c96c5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.025883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21ffcd-9148-46f9-a284-83646c96c5db admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.026163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21ffcd-9148-46f9-a284-83646c96c5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.026466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21ffcd-9148-46f9-a284-83646c96c5db admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.027105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21ffcd-9148-46f9-a284-83646c96c5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.029301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21ffcd-9148-46f9-a284-83646c96c5db admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.039600 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1321/2673] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:54.043780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.054102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc8865-ef21-43ba-b662-36a67c73f1d2 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.058610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.059509 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.059822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.060150 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] PATH_INFO: `/v3/projects/35d339ab000f4ac0acce53e5cfaf45eb/users/b822ec7da34f4bfd875fcf70c07ae9c7/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.074569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d339ab000f4ac0acce53e5cfaf45eb, user_id=b822ec7da34f4bfd875fcf70c07ae9c7, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.078341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.149081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1353/2674] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:54 2022] PUT /identity/v3/projects/35d339ab000f4ac0acce53e5cfaf45eb/users/b822ec7da34f4bfd875fcf70c07ae9c7/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:54.160157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.160444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.160721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.187989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] MFA Rules not processed for user `b822ec7da34f4bfd875fcf70c07ae9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:54.189247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:54.253257 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1322/2675] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:54.257875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.319371 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae11a1-3a3e-4116-a671-ace56747183a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.323832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.324425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.324702 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.325518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.326245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.328130 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.442441 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1354/2676] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:54 2022] POST /identity/v3/projects => generated 371 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:54.446999 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.461703 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df969e-6726-43aa-a90f-705ddb26a22e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.526804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.527408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.527686 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.528000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.528838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.530778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.686800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1323/2677] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:54 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:54.691049 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.702276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbadec-1ff4-40f2-8231-c7758154555e admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.706902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.707501 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.707779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.708055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.708686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.710319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.721269 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1355/2678] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:54.725386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.735555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec870d84-7f1d-47c6-b25c-11ded9f10abc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.740163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.740809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.741126 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.741399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] PATH_INFO: `/v3/projects/d9da799ca2f3403781cdd49c1a7e8ff9/users/1262cd2642fa4bfd90cdc8eb3fcc8465/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.756321 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9da799ca2f3403781cdd49c1a7e8ff9, user_id=1262cd2642fa4bfd90cdc8eb3fcc8465, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.759076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.829309 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1324/2679] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:54 2022] PUT /identity/v3/projects/d9da799ca2f3403781cdd49c1a7e8ff9/users/1262cd2642fa4bfd90cdc8eb3fcc8465/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:54.833426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.843863 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43692f-4aa2-4d13-bba5-8fe9678ec486 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.876536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:54.899266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08559cdb-97c2-4a76-b814-f729c5ebc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.899864 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08559cdb-97c2-4a76-b814-f729c5ebc086 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.900137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08559cdb-97c2-4a76-b814-f729c5ebc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.900407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08559cdb-97c2-4a76-b814-f729c5ebc086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.901055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08559cdb-97c2-4a76-b814-f729c5ebc086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.902680 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08559cdb-97c2-4a76-b814-f729c5ebc086 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.912097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1356/2680] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:42:54.916189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:54.922052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:54.929641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ad571-a545-42a7-b170-bd0be115dcdd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:54.931932 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:54.932571 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.932846 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.933145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] PATH_INFO: `/v3/projects/d9da799ca2f3403781cdd49c1a7e8ff9/users/1262cd2642fa4bfd90cdc8eb3fcc8465/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.935785 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9da799ca2f3403781cdd49c1a7e8ff9, user_id=1262cd2642fa4bfd90cdc8eb3fcc8465, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:54.938435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:54.948463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1325/2681] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:54 2022] PUT /identity/v3/projects/d9da799ca2f3403781cdd49c1a7e8ff9/users/1262cd2642fa4bfd90cdc8eb3fcc8465/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:54.959629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:54.963986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:54.963986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:54.988532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] MFA Rules not processed for user `1262cd2642fa4bfd90cdc8eb3fcc8465`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:54.992632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:55.060638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1357/2682] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:55.071805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:55.074228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:55.133279 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d665043-b989-46dd-9f30-0af30f48c373 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:55.136917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29009ccd-cd2b-40b9-9ab2-10ae715187b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:55.137553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29009ccd-cd2b-40b9-9ab2-10ae715187b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:55.137826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29009ccd-cd2b-40b9-9ab2-10ae715187b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:55.138211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29009ccd-cd2b-40b9-9ab2-10ae715187b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:55.147569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29009ccd-cd2b-40b9-9ab2-10ae715187b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:55.149065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29009ccd-cd2b-40b9-9ab2-10ae715187b7 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:55.160412 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1326/2683] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:56.098017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:56.163162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78056cff-182b-42f0-afe9-a3d4b85d5b48 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:56.167694 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:56.168329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:56.168847 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:56.169166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] PATH_INFO: `/v3/users/b822ec7da34f4bfd875fcf70c07ae9c7` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:56.170294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b822ec7da34f4bfd875fcf70c07ae9c7)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:56.172238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:56.420121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b822ec7da34f4bfd875fcf70c07ae9c7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:56.420777 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b822ec7da34f4bfd875fcf70c07ae9c7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:56.427884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b822ec7da34f4bfd875fcf70c07ae9c7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:56.428520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b822ec7da34f4bfd875fcf70c07ae9c7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:56.431099 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:56.548436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b822ec7da34f4bfd875fcf70c07ae9c7'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:56.566800 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1358/2684] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:56 2022] DELETE /identity/v3/users/b822ec7da34f4bfd875fcf70c07ae9c7 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:56.571240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:56.571762 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:56.571874 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:56.591103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:56.592338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:56.652528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1327/2685] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:56.663028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:56.664759 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:56.722492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9e7ed-87db-4db8-8eea-ae220f6d4378 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:56.726137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:56.726738 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:56.727017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:56.727288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:56.736999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:56.738424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:56.747894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1359/2686] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:42:57.209264 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:57.210766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:57.224662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91ccc-6eb6-421f-95f9-80e65f2dc9cf None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:57.230095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:57.230731 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:57.231036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:57.231907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] PATH_INFO: `/v3/projects/35d339ab000f4ac0acce53e5cfaf45eb` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:57.233177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d339ab000f4ac0acce53e5cfaf45eb)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:57.235070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:57.360689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d339ab000f4ac0acce53e5cfaf45eb', 'request_id': 'req-460d3f17-9da4-4373-ab47-3a08d9da3037'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.366870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d339ab000f4ac0acce53e5cfaf45eb', 'request_id': 'req-460d3f17-9da4-4373-ab47-3a08d9da3037'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.372568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d339ab000f4ac0acce53e5cfaf45eb', 'request_id': 'req-460d3f17-9da4-4373-ab47-3a08d9da3037'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.422913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] The token cache is being invalidate because project 35d339ab000f4ac0acce53e5cfaf45eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:57.423233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.423856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.425371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1328/2687] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:57 2022] DELETE /identity/v3/projects/35d339ab000f4ac0acce53e5cfaf45eb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:57.429364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:57.440144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cc8ce-8808-4d77-86d8-d85a182c2fe3 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:57.508453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:57.509131 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:57.509416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:57.509852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] PATH_INFO: `/v3/users/1262cd2642fa4bfd90cdc8eb3fcc8465` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:57.511071 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1262cd2642fa4bfd90cdc8eb3fcc8465)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:57.513650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:57.751844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1262cd2642fa4bfd90cdc8eb3fcc8465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.752414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1262cd2642fa4bfd90cdc8eb3fcc8465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.759879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1262cd2642fa4bfd90cdc8eb3fcc8465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.760523 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1262cd2642fa4bfd90cdc8eb3fcc8465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.851553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-84650289-7179-4999-8c96-5bd6f4155472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1262cd2642fa4bfd90cdc8eb3fcc8465'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:57.865601 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:57.870610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1360/2688] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:42:57 2022] DELETE /identity/v3/users/1262cd2642fa4bfd90cdc8eb3fcc8465 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:58.287292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:58.288891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:58.309930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d3f17-9da4-4373-ab47-3a08d9da3037 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:58.321739 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:58.368707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:58.369483 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:58.369759 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:58.370034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] PATH_INFO: `/v3/projects/d9da799ca2f3403781cdd49c1a7e8ff9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:58.371184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9da799ca2f3403781cdd49c1a7e8ff9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:58.373949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:58.579058 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9da799ca2f3403781cdd49c1a7e8ff9', 'request_id': 'req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:58.587779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9da799ca2f3403781cdd49c1a7e8ff9', 'request_id': 'req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:58.598460 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9da799ca2f3403781cdd49c1a7e8ff9', 'request_id': 'req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:58.719899 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] The token cache is being invalidate because project d9da799ca2f3403781cdd49c1a7e8ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:42:58.720189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:58.720809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-02ff6fa2-dc8c-4a47-acea-b08e3fcb6565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:42:58.722311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1329/2689] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:42:58 2022] DELETE /identity/v3/projects/d9da799ca2f3403781cdd49c1a7e8ff9 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:58.727178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:58.727712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:58.727712 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:58.749791 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:58.751224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:58.810554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1361/2690] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:58.818423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:58.818741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:58.819019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:58.847664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:58.848963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:58.859562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1330/2691] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:58.863903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:58.875852 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca7811-4f7a-4077-9c55-0f23ab1e045f None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:58.881355 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:58.881958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:58.882237 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:58.882510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:58.883153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:42:58.883455 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:58.885105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:58.896679 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1362/2692] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:42:58.901106 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:58.911301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a5da1-1226-44c1-a64b-473a262e5ca8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:58.916060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:58.916657 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:58.916969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:58.917248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:58.917912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:58.920523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.013385 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1331/2693] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:58 2022] POST /identity/v3/projects => generated 399 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:59.018400 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.028993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab27e6-597e-4e12-a617-aea3f4695c42 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:59.085089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:59.085619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.085898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.086426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.087048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:59.088981 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.246080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1363/2694] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:59 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:59.250249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.260342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc72052-ba40-4b29-b629-40abdba200b1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:59.266008 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:59.266603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.266877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.267147 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.267779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:59.269401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.280227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1332/2695] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:59.284346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.294344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3b02c-c34c-4a8a-9f89-fc1456d04283 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:59.298867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40b7bb-8566-4d88-8b1c-854438257c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:59.299452 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40b7bb-8566-4d88-8b1c-854438257c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.299729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40b7bb-8566-4d88-8b1c-854438257c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.300001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40b7bb-8566-4d88-8b1c-854438257c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.300636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40b7bb-8566-4d88-8b1c-854438257c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:59.302333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40b7bb-8566-4d88-8b1c-854438257c06 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.312895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1364/2696] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:42:59.316709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.327557 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f986eb5-8599-4346-856a-de9ea93472ec admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:59.332210 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:59.332930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.333173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.333447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] PATH_INFO: `/v3/projects/b0e9088ea24345c5904e7bd879e65e40/users/a0562a3cb55949cd88775c8574171626/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.349595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e9088ea24345c5904e7bd879e65e40, user_id=a0562a3cb55949cd88775c8574171626, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:59.352258 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.451839 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1333/2697] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:42:59 2022] PUT /identity/v3/projects/b0e9088ea24345c5904e7bd879e65e40/users/a0562a3cb55949cd88775c8574171626/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:42:59.462642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.462909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.463184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.494207 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] MFA Rules not processed for user `a0562a3cb55949cd88775c8574171626`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:42:59.495456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:42:59.548992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:59.558632 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1365/2698] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:42:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:42:59.563406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.608564 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:59.627103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf34d9-8272-488a-901f-5210ea38efb5 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:59.631798 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:59.632461 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.632744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.633053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.633698 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:59.635607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.795941 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1334/2699] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:42:59 2022] POST /identity/v3/projects => generated 401 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:59.800330 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.811887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fc713-28f4-4a48-a3e3-5c9808f4d863 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:59.868933 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:59.869478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:59.869761 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:59.870040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:59.870668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:59.872812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:59.983858 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1366/2700] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:42:59 2022] POST /identity/v3/users => generated 307 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:42:59.987903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:59.998329 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fdbb1-7a55-4271-b1ae-f9d350d0cfa0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:00.003019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:00.003654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.003969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.004241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.004894 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:00.006491 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:00.017451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1335/2701] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:00.021591 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:00.032358 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8312a-78f8-4a31-906a-8aabda84744b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:00.036962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:00.040173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.040173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.040173 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] PATH_INFO: `/v3/projects/7e24d97e9b574025b230303802c3fac5/users/bd87a3332f074840951cc07046f4e485/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.052771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e24d97e9b574025b230303802c3fac5, user_id=bd87a3332f074840951cc07046f4e485, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:00.055631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:00.155773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1367/2702] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:00 2022] PUT /identity/v3/projects/7e24d97e9b574025b230303802c3fac5/users/bd87a3332f074840951cc07046f4e485/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:00.159858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:00.170262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb630525-0ab7-4872-8049-f3d783276079 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:00.228760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c26148-4c58-401e-9647-4b588832eae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:00.229413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c26148-4c58-401e-9647-4b588832eae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.229674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c26148-4c58-401e-9647-4b588832eae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.229945 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c26148-4c58-401e-9647-4b588832eae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.230637 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c26148-4c58-401e-9647-4b588832eae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:00.232767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c26148-4c58-401e-9647-4b588832eae6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:00.242523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1336/2703] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:00.247142 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:00.257369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5226e-7bdf-404a-8f75-685837318167 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:00.261989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:00.263646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.263920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.264190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] PATH_INFO: `/v3/projects/7e24d97e9b574025b230303802c3fac5/users/bd87a3332f074840951cc07046f4e485/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.266186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e24d97e9b574025b230303802c3fac5, user_id=bd87a3332f074840951cc07046f4e485, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:00.268827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:00.281873 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1368/2704] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:00 2022] PUT /identity/v3/projects/7e24d97e9b574025b230303802c3fac5/users/bd87a3332f074840951cc07046f4e485/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:00.307593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.307593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.307593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.323035 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] MFA Rules not processed for user `bd87a3332f074840951cc07046f4e485`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:00.326914 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:00.395590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1337/2705] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:00 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:00.407624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:00.409475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:00.467636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb242938-a903-4e0d-8904-ce66155ff5a1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:00.471286 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:00.471880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.472155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.472426 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.481560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:00.482983 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:00.492349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1369/2706] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:00 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:00.729715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:00.739998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6461e4a-c86e-43ad-91e8-7b87b415eaea None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:00.743579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b13cd2-643c-4ccf-a8ce-06cdf55f4163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:00.744175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13cd2-643c-4ccf-a8ce-06cdf55f4163 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:00.744455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13cd2-643c-4ccf-a8ce-06cdf55f4163 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:00.744729 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b13cd2-643c-4ccf-a8ce-06cdf55f4163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:00.777590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:00.800191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13cd2-643c-4ccf-a8ce-06cdf55f4163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:00.803422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b13cd2-643c-4ccf-a8ce-06cdf55f4163 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:00.810807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1338/2707] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:10.488217 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:10.856985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=11 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:12.238004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:12.345886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3dfb68-98b7-4ade-91e5-748006b5b307 service cinder] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:12.350507 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:12.351137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:12.351415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:12.351689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] PATH_INFO: `/v3/users/a0562a3cb55949cd88775c8574171626` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:12.352821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0562a3cb55949cd88775c8574171626)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:12.354786 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:12.557843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0562a3cb55949cd88775c8574171626'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:12.558503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0562a3cb55949cd88775c8574171626'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:12.566571 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0562a3cb55949cd88775c8574171626'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:12.567262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0562a3cb55949cd88775c8574171626'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:12.651435 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0562a3cb55949cd88775c8574171626'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:12.670244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1370/2708] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:12 2022] DELETE /identity/v3/users/a0562a3cb55949cd88775c8574171626 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:12.675501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:12.676083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:12.676083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:12.729229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:12.730436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:12.789107 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1339/2709] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:12.799066 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:12.800472 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:12.841765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:12.876028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741a654-92b1-4d4a-8c2e-fb48266cc65a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:12.879757 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:12.880423 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:12.880736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:12.881036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:12.893044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:12.901805 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:12.905033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1371/2710] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:13.363744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:13.365641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:13.378538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de593fa-6042-453a-9e0b-dda7ceb83ee4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:13.383277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:13.383910 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:13.388356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:13.388356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] PATH_INFO: `/v3/projects/b0e9088ea24345c5904e7bd879e65e40` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:13.388356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e9088ea24345c5904e7bd879e65e40)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:13.388356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:13.538354 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e9088ea24345c5904e7bd879e65e40', 'request_id': 'req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.544913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e9088ea24345c5904e7bd879e65e40', 'request_id': 'req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.551671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e9088ea24345c5904e7bd879e65e40', 'request_id': 'req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.629012 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] The token cache is being invalidate because project b0e9088ea24345c5904e7bd879e65e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:13.629312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.629949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.631449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1340/2711] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:13 2022] DELETE /identity/v3/projects/b0e9088ea24345c5904e7bd879e65e40 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:13.635486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:13.645353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd596e8-9b45-4610-a95a-2499a3b88bd0 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:13.702265 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:13.702929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:13.703278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:13.703555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] PATH_INFO: `/v3/users/bd87a3332f074840951cc07046f4e485` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:13.704686 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd87a3332f074840951cc07046f4e485)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:13.706636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:13.896245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd87a3332f074840951cc07046f4e485'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.897029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd87a3332f074840951cc07046f4e485'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.904714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd87a3332f074840951cc07046f4e485'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.905425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd87a3332f074840951cc07046f4e485'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:13.994333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ecac82ba-e881-427f-9401-58dc776d0e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd87a3332f074840951cc07046f4e485'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:14.011908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1372/2712] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:13 2022] DELETE /identity/v3/users/bd87a3332f074840951cc07046f4e485 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:14.442606 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:14.444141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:14.457068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0c35d-9e59-4b68-8ef2-33d5edb95cfc admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:14.527113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:14.536342 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:14.537024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:14.538260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:14.538599 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] PATH_INFO: `/v3/projects/7e24d97e9b574025b230303802c3fac5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:14.540034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e24d97e9b574025b230303802c3fac5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:14.542223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:14.692398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e24d97e9b574025b230303802c3fac5', 'request_id': 'req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:14.698553 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e24d97e9b574025b230303802c3fac5', 'request_id': 'req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:14.709413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e24d97e9b574025b230303802c3fac5', 'request_id': 'req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:14.800641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] The token cache is being invalidate because project 7e24d97e9b574025b230303802c3fac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:14.800962 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:14.801596 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-4ace2cb1-41ea-4e7d-9160-d52ef768ecfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:14.803095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1341/2713] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:14 2022] DELETE /identity/v3/projects/7e24d97e9b574025b230303802c3fac5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:14.807885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:14.808382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:14.808481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:14.860758 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:14.862081 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:14.875942 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:14.921471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1373/2714] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:14.930516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:14.935339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:14.935339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:14.951521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:14.952697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:14.962054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1342/2715] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:14.966515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:14.977804 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fa00b-3520-4e4b-b2a9-d3dd0ea820d1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:14.982722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:14.983322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:14.987038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:14.987038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:14.987038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:14.987038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:14.987038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:14.997823 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1374/2716] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:43:15.002646 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.011758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141da9-5b7a-47dc-bb88-ca8079605b72 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.017191 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.017788 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.018064 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.018328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.018953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.020830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:15.116231 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1343/2717] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:15 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:15.120561 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.131296 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28154864-ccfa-41ea-abb9-e0649c0f3a90 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.190914 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.191581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.191869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.192179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.192824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.194949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:15.274743 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1375/2718] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:15 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:15.279603 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.290982 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9fc60-eb3b-4f11-8ca4-ed32b5cedfbe admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.295737 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.296331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.297143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.297415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.298051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.299629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:15.309398 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1344/2719] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:15.314522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.324444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5bc9-06b3-40ee-8752-1ca31cb28a77 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.329515 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f26b7-b71c-4410-b91c-6710bc87cc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.330106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f26b7-b71c-4410-b91c-6710bc87cc19 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.333741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f26b7-b71c-4410-b91c-6710bc87cc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.333741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f26b7-b71c-4410-b91c-6710bc87cc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.333741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f26b7-b71c-4410-b91c-6710bc87cc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.333741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f26b7-b71c-4410-b91c-6710bc87cc19 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:15.344295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1376/2720] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:15.348291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.358403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6791fa-beea-4179-b401-53e56fde661b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.364010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.367299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.367299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.367299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] PATH_INFO: `/v3/projects/412363788efc485d87acae6c824e0851/users/7096ed4d99874089bad47e7cbf3e0bf9/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.380995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=412363788efc485d87acae6c824e0851, user_id=7096ed4d99874089bad47e7cbf3e0bf9, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.384053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:15.493365 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1345/2721] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:15 2022] PUT /identity/v3/projects/412363788efc485d87acae6c824e0851/users/7096ed4d99874089bad47e7cbf3e0bf9/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:15.504841 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.505181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.505459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.534166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] MFA Rules not processed for user `7096ed4d99874089bad47e7cbf3e0bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:15.535377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:15.606080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1377/2722] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:15.618315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.620567 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:15.680255 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307ae22-20db-4f9d-85e0-96ca31ee94ce admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.683915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275163dd-d395-4592-b4dd-d820060032bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.684515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275163dd-d395-4592-b4dd-d820060032bb service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.684792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275163dd-d395-4592-b4dd-d820060032bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.685104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275163dd-d395-4592-b4dd-d820060032bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.693957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275163dd-d395-4592-b4dd-d820060032bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.695372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275163dd-d395-4592-b4dd-d820060032bb service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:15.705142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1346/2723] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:15.832033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:15.835526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:15.855517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:15.909311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e6f4c-b81e-4cfc-872d-bc52ee209620 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:15.913772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:15.914404 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:15.914684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:15.914960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] PATH_INFO: `/v3/users/ca0e8a9fc65b4c34b7c7eb2d81bb9136` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:15.916092 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0e8a9fc65b4c34b7c7eb2d81bb9136)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:15.918088 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:16.122045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0e8a9fc65b4c34b7c7eb2d81bb9136'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:16.122820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0e8a9fc65b4c34b7c7eb2d81bb9136'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:16.133063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0e8a9fc65b4c34b7c7eb2d81bb9136'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:16.133973 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0e8a9fc65b4c34b7c7eb2d81bb9136'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:16.227798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0e8a9fc65b4c34b7c7eb2d81bb9136'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:16.248624 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1378/2724] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:15 2022] DELETE /identity/v3/users/ca0e8a9fc65b4c34b7c7eb2d81bb9136 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:16.253643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:16.253925 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:16.254196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:16.273930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:16.275117 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:16.294955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:16.338157 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1347/2725] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:16.348267 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:16.350082 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:16.414731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fde753-85a2-4146-bcee-1175df4bc584 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:16.418409 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:16.419009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:16.419282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:16.419553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:16.429128 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:16.431176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:16.440580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1379/2726] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:43:16.842614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:16.845187 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:16.857444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7999-e0b1-4d08-9ff8-de9a46ffc1f8 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:16.863593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:16.864233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:16.864505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:16.864769 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] PATH_INFO: `/v3/users/389ed248f4974d3dbca8ef00086782ad` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:16.866024 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389ed248f4974d3dbca8ef00086782ad)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:16.867996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:16.989668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:16.991174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:17.004444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929931e5-787e-436a-a6b3-c02e28725481 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:17.010845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:17.011513 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:17.011795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:17.012111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] PATH_INFO: `/v3/projects/a0a1778b4d914492ab96929aa2e0984e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:17.013425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a1778b4d914492ab96929aa2e0984e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:17.015487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:17.069904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389ed248f4974d3dbca8ef00086782ad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.070625 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389ed248f4974d3dbca8ef00086782ad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.078235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389ed248f4974d3dbca8ef00086782ad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.079471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389ed248f4974d3dbca8ef00086782ad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.097310 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:17.118586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a1778b4d914492ab96929aa2e0984e', 'request_id': 'req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.124514 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a1778b4d914492ab96929aa2e0984e', 'request_id': 'req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.136676 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f3b58527-6cbf-4538-bb14-c65f3525ded6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389ed248f4974d3dbca8ef00086782ad'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.154972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1348/2727] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:16 2022] DELETE /identity/v3/users/389ed248f4974d3dbca8ef00086782ad => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:17.159741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:17.160023 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:17.160343 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:17.179417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:17.180608 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:17.205428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a1778b4d914492ab96929aa2e0984e', 'request_id': 'req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.217030 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] The token cache is being invalidate because project a0a1778b4d914492ab96929aa2e0984e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:17.217322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.217989 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.220116 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1380/2728] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:16 2022] DELETE /identity/v3/projects/a0a1778b4d914492ab96929aa2e0984e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:17.224183 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:17.225822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:17.240407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1349/2729] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:17.251495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:17.253015 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:17.297700 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cbec1-ea6e-46dd-bb2d-a1bcfc0d93bb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:17.299943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:17.302710 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:17.303347 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:17.303683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:17.303945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] PATH_INFO: `/v3/users/23c655cc103b4d38a6d9cce43bee0ef2` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:17.305122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c655cc103b4d38a6d9cce43bee0ef2)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:17.307200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:17.313790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb481-01df-4fd4-b806-a06c8b8e411a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:17.317463 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:17.318087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:17.318350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:17.318615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:17.328247 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:17.329987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:17.340427 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1350/2730] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:43:17.439039 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c655cc103b4d38a6d9cce43bee0ef2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.439716 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c655cc103b4d38a6d9cce43bee0ef2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.446986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c655cc103b4d38a6d9cce43bee0ef2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.447641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c655cc103b4d38a6d9cce43bee0ef2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.503885 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c655cc103b4d38a6d9cce43bee0ef2'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:17.548722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1381/2731] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:17 2022] DELETE /identity/v3/users/23c655cc103b4d38a6d9cce43bee0ef2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:17.882989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:17.885371 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:17.900010 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897b3bb-3473-4852-ad9c-3ac91e97938d service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:17.978845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:17.979503 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:17.979867 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:17.980218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] PATH_INFO: `/v3/projects/377e2eda387845078df49f56396654d6` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:17.981464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=377e2eda387845078df49f56396654d6)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:17.983462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.143721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377e2eda387845078df49f56396654d6', 'request_id': 'req-0a7d3f7f-642d-4ea8-8642-02d967cf6838'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.149825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377e2eda387845078df49f56396654d6', 'request_id': 'req-0a7d3f7f-642d-4ea8-8642-02d967cf6838'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.155249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377e2eda387845078df49f56396654d6', 'request_id': 'req-0a7d3f7f-642d-4ea8-8642-02d967cf6838'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.175468 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.176991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:18.194356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f40700-a84d-42a9-badd-6358542496c3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.243115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:18.259015 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.259650 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.259934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.260211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] PATH_INFO: `/v3/projects/c69878eaff174ad9885ce990611d2881` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.261414 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69878eaff174ad9885ce990611d2881)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.263337 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.273132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] The token cache is being invalidate because project 377e2eda387845078df49f56396654d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:18.277539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.277539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-0a7d3f7f-642d-4ea8-8642-02d967cf6838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.277539 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1351/2732] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:17 2022] DELETE /identity/v3/projects/377e2eda387845078df49f56396654d6 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:18.280254 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b5dcd-721e-4b73-8d42-3fc22850793f None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.280543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b5dcd-721e-4b73-8d42-3fc22850793f None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.280816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b5dcd-721e-4b73-8d42-3fc22850793f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.301518 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-6d5b5dcd-721e-4b73-8d42-3fc22850793f None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:18.302901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6d5b5dcd-721e-4b73-8d42-3fc22850793f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:18.310974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1352/2733] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:18.319771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.320053 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.320324 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.340136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:18.341316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:18.350362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1353/2734] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:18.354631 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.364907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c5799-7107-4a3f-acb0-d1f3913097aa None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.369529 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.380120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.380120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.380120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.380120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:18.380120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.380120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.385050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1354/2735] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:43:18.389512 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.399550 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b669f-9c07-4f6f-9947-fea7815adff3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.404271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.414638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.414638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.414638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.414638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.414638 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.417727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69878eaff174ad9885ce990611d2881', 'request_id': 'req-7e383e0d-d1b1-49aa-9475-52d70bbf243f'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.424729 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69878eaff174ad9885ce990611d2881', 'request_id': 'req-7e383e0d-d1b1-49aa-9475-52d70bbf243f'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.504338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1355/2736] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:43:18.508392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.525656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69878eaff174ad9885ce990611d2881', 'request_id': 'req-7e383e0d-d1b1-49aa-9475-52d70bbf243f'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.536818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:18.538114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] The token cache is being invalidate because project c69878eaff174ad9885ce990611d2881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:18.538474 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.539177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-7e383e0d-d1b1-49aa-9475-52d70bbf243f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:18.540776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1382/2737] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:18 2022] DELETE /identity/v3/projects/c69878eaff174ad9885ce990611d2881 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:18.545500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91053-656a-4c82-af66-a566e14910fe None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.545855 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91053-656a-4c82-af66-a566e14910fe None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.546228 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d91053-656a-4c82-af66-a566e14910fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.567384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-16d91053-656a-4c82-af66-a566e14910fe None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:18.568662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-16d91053-656a-4c82-af66-a566e14910fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:18.580543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df8d74-0465-4ab9-8f1a-5018a4826fd0 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.585615 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.586230 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.587055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.587353 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.587993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.589968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.631636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1383/2738] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:18.639448 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.639734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.640012 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.660616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:18.661879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:18.669753 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1384/2739] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:18.673727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.682498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9d3b4-3c5b-4bb6-b4a0-e47acc13d6f5 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.687271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.687836 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.688122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.688396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.689194 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:18.689504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.691172 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.702312 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1385/2740] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:43:18.706943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.716991 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fde0b4-b687-43bd-8841-36fe7dfe67cd admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.722055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.722617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.722893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.723166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.723915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.725812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.748780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1356/2741] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:43:18.752993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.754679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:18.766250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b132b-45e5-4049-83a1-49c2db92791c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.771446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.772016 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.772291 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.772559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.773455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.775093 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.785458 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1357/2742] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:18.789131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.798732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7be50e-c2bc-4bf2-896a-8fe6067275c3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.803223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.803800 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.804078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.804347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.804979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.808092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.816713 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1358/2743] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:18.820590 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.830401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15a3f0-f7cc-4999-aea4-67a550c93c51 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.835100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91756341-9eb7-440a-a96a-67c6430ea16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.839148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1386/2744] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:43:18.839337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756341-9eb7-440a-a96a-67c6430ea16c admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.839337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756341-9eb7-440a-a96a-67c6430ea16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.839337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91756341-9eb7-440a-a96a-67c6430ea16c admin admin] PATH_INFO: `/v3/projects/eed55f7ab39c4a60ab366e77e0e1d7a0/users/56bfa1688ede455e9ec01c5b274747c1/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.840424 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:18.852209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91756341-9eb7-440a-a96a-67c6430ea16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed55f7ab39c4a60ab366e77e0e1d7a0, user_id=56bfa1688ede455e9ec01c5b274747c1, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.854896 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91756341-9eb7-440a-a96a-67c6430ea16c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.904575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c8529-51f1-4fc2-84a6-25a61e8e6985 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:18.910169 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:18.910778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.911059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.911333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.911970 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:18.913934 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:18.931410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1359/2745] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:18 2022] PUT /identity/v3/projects/eed55f7ab39c4a60ab366e77e0e1d7a0/users/56bfa1688ede455e9ec01c5b274747c1/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:18.941807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:18.942087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:18.942361 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:18.942948 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:18.970773 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] MFA Rules not processed for user `56bfa1688ede455e9ec01c5b274747c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:18.971972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:19.033198 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1360/2746] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:19.037482 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1387/2747] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:18 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:43:19.041097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.045278 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.046952 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:19.103516 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c615b-6ba8-4519-9f2e-159dd0047f09 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:19.103969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5b4e4-c18e-4852-a5e7-696d7bd0315f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:19.108318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:19.108955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.109238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.109514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.110164 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:19.112050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:19.128654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1361/2748] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:19.129790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.144758 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af921e1-7b68-492e-91c7-a32f8e4a749f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:19.149883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:19.150531 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.150809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.151182 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.151828 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:19.153477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:19.162154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c815b-f23e-4770-940b-b9aa0cd89c5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:19.162846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c815b-f23e-4770-940b-b9aa0cd89c5f service cinder] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.163197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c815b-f23e-4770-940b-b9aa0cd89c5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.163540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c815b-f23e-4770-940b-b9aa0cd89c5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.164193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1362/2749] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:19.168250 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.172126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c815b-f23e-4770-940b-b9aa0cd89c5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:19.174224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c815b-f23e-4770-940b-b9aa0cd89c5f service cinder] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:19.178372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6fb75-cbdf-4514-82ac-bbc01da57341 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:19.183186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1388/2750] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:19.186240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:19.186240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.186240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.186240 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] PATH_INFO: `/v3/projects/3a03dd42ba4249a389585bb1f421482a/users/26292b5ea3224309a27acc13b9657324/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.200903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a03dd42ba4249a389585bb1f421482a, user_id=26292b5ea3224309a27acc13b9657324, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:19.203582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:19.208709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:19.303222 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1363/2751] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:19 2022] PUT /identity/v3/projects/3a03dd42ba4249a389585bb1f421482a/users/26292b5ea3224309a27acc13b9657324/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:19.315381 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.315667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.315944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.344197 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] MFA Rules not processed for user `26292b5ea3224309a27acc13b9657324`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:19.346597 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:19.410215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1389/2752] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:19.421286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.423507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:19.478934 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4593d-774c-4e7e-8623-15c0c087b308 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:19.482581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ab90a-ddd1-4e54-a6e5-cea97bf2b421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:19.483175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ab90a-ddd1-4e54-a6e5-cea97bf2b421 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.483452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ab90a-ddd1-4e54-a6e5-cea97bf2b421 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.483721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ab90a-ddd1-4e54-a6e5-cea97bf2b421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.492578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ab90a-ddd1-4e54-a6e5-cea97bf2b421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:19.494036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ab90a-ddd1-4e54-a6e5-cea97bf2b421 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:19.503805 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1364/2753] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:19.522911 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.531808 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:19.587063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d2f3a-b15e-4a93-8e3f-554a872ff218 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:19.591593 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:19.592324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.592608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.592910 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] PATH_INFO: `/v3/users/26292b5ea3224309a27acc13b9657324` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.594046 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26292b5ea3224309a27acc13b9657324)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:19.596272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:19.762341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26292b5ea3224309a27acc13b9657324'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:19.763010 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26292b5ea3224309a27acc13b9657324'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:19.771151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26292b5ea3224309a27acc13b9657324'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:19.771993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26292b5ea3224309a27acc13b9657324'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:19.816947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26292b5ea3224309a27acc13b9657324'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:19.849350 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1390/2754] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:19 2022] DELETE /identity/v3/users/26292b5ea3224309a27acc13b9657324 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:19.853832 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:19.854125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:19.854402 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:19.880236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:19.880236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:19.940223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1365/2755] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:19.950000 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:19.951408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:20.010627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c870666-6710-4a24-98ed-3a043192c887 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:20.014327 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53a68b-4fe5-4466-b746-0c8d2db451ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:20.014961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53a68b-4fe5-4466-b746-0c8d2db451ec service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:20.015241 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53a68b-4fe5-4466-b746-0c8d2db451ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:20.015519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53a68b-4fe5-4466-b746-0c8d2db451ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:20.024648 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53a68b-4fe5-4466-b746-0c8d2db451ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:20.027040 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53a68b-4fe5-4466-b746-0c8d2db451ec service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:20.036243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1391/2756] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:20.406858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:20.413705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:20.421863 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4113803-53e1-4bfc-9997-8b4db598ece9 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:20.427464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:20.428097 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:20.428372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:20.428643 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] PATH_INFO: `/v3/projects/3a03dd42ba4249a389585bb1f421482a` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:20.429892 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a03dd42ba4249a389585bb1f421482a)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:20.432032 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:20.554335 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a03dd42ba4249a389585bb1f421482a', 'request_id': 'req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:20.560474 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a03dd42ba4249a389585bb1f421482a', 'request_id': 'req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:20.562444 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:20.566498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a03dd42ba4249a389585bb1f421482a', 'request_id': 'req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:20.656156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] The token cache is being invalidate because project 3a03dd42ba4249a389585bb1f421482a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:20.656383 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:20.657034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-fba6d72a-2586-4e7f-a2d9-ba0c7552ca89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:20.667055 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1366/2757] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:20 2022] DELETE /identity/v3/projects/3a03dd42ba4249a389585bb1f421482a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:20.667209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:20.667209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:20.667209 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:20.686373 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:20.687581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:20.711999 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:20.747532 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1392/2758] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:20.755597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:20.755886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:20.756162 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:20.776446 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:20.781559 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:20.786475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1367/2759] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:20.790663 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:20.800769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ada1b-2152-4aaa-943b-a22275dd3228 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:20.805626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:20.806230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:20.806826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:20.807229 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:20.807869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:20.808202 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:20.809903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:20.821232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1393/2760] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:43:20.825367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:20.835103 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf6f83-c728-44cb-84bc-fc34ab95ae3d None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:20.840178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:20.840804 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:20.841116 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:20.841393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:20.842030 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:20.843949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:20.989946 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1368/2761] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:20 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:20.994153 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:21.004129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82b2fc-9725-4069-ba20-ec7dcecde617 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:21.060528 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:21.061159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.061439 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.061714 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.062344 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:21.064453 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:21.181176 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1394/2762] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:20 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:21.181869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:21.192449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48884ee2-4c63-4177-9c85-61adc29232f4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:21.197189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:21.197833 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.198115 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.198390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.199068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:21.200674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:21.211883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1369/2763] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:21.215869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:21.226633 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc028538-d0e3-4486-9427-ecca1c285e33 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:21.231510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80b1b0-de31-4a1d-90a3-53fea3f598a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:21.232107 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80b1b0-de31-4a1d-90a3-53fea3f598a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.232388 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80b1b0-de31-4a1d-90a3-53fea3f598a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.232665 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80b1b0-de31-4a1d-90a3-53fea3f598a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.233341 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80b1b0-de31-4a1d-90a3-53fea3f598a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:21.234938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80b1b0-de31-4a1d-90a3-53fea3f598a8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:21.244792 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1395/2764] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:21.249196 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:21.259663 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaad69e-d80f-45cb-b4e2-83c2410f9698 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:21.264139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:21.264780 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.265088 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.265362 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] PATH_INFO: `/v3/projects/8995416d93f5422a9ca6213641b93dad/users/27a74a36c48148e0ba0be713cbe4aa11/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.280382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8995416d93f5422a9ca6213641b93dad, user_id=27a74a36c48148e0ba0be713cbe4aa11, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:21.283067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:21.359351 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1370/2765] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:21 2022] PUT /identity/v3/projects/8995416d93f5422a9ca6213641b93dad/users/27a74a36c48148e0ba0be713cbe4aa11/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:21.369888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e335040-3732-4aa0-9572-f5efcfad8bda None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.370179 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e335040-3732-4aa0-9572-f5efcfad8bda None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.370459 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e335040-3732-4aa0-9572-f5efcfad8bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.398875 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-3e335040-3732-4aa0-9572-f5efcfad8bda None None] MFA Rules not processed for user `27a74a36c48148e0ba0be713cbe4aa11`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:21.400136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3e335040-3732-4aa0-9572-f5efcfad8bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:21.465616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1396/2766] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:21.476662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:21.479017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:21.539579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5b57e-e6fc-4c27-996b-9fce1e50def2 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:21.543378 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:21.544018 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.544298 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.544569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.553712 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:21.555144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:21.564977 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1371/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:43:21.971538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:21.972059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:21.972059 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:21.992138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] MFA Rules not processed for user `56bfa1688ede455e9ec01c5b274747c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:21.993463 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:22.012232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:22.050331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1397/2768] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:26.153688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:26.155140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:26.245835 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e100dcb-3ffb-485a-a7a3-abc0def3b0bd service glance] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:26.251514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:26.252154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:26.252437 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:26.252714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:26.254502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:26.263655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:26.269533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:26.271602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d2b3-95e3-4ac1-ba23-3a6f3ca9ea88 service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:26.285265 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1372/2769] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:32.588583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=14 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:36.519520 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:41.658397 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:41.659845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:41.733511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7b7e0-1f2c-4994-9998-c97fd78aee8b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:41.741979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:41.742683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:41.743027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:41.743365 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] PATH_INFO: `/v3/users/7096ed4d99874089bad47e7cbf3e0bf9` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:41.744605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7096ed4d99874089bad47e7cbf3e0bf9)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:41.746770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:42.027343 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7096ed4d99874089bad47e7cbf3e0bf9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.027867 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7096ed4d99874089bad47e7cbf3e0bf9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.041484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7096ed4d99874089bad47e7cbf3e0bf9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.041484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7096ed4d99874089bad47e7cbf3e0bf9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.124320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7096ed4d99874089bad47e7cbf3e0bf9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.144477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1398/2770] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:41 2022] DELETE /identity/v3/users/7096ed4d99874089bad47e7cbf3e0bf9 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:42.149194 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:42.149704 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:42.149807 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:42.169740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:42.170953 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:42.230802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1373/2771] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:42.241133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:42.247554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:42.342617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabd819-fce6-483c-ad4d-93e1b46fdec3 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:42.346317 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa13763-2f83-4ee5-a1e5-d8c38b80d4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:42.352518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13763-2f83-4ee5-a1e5-d8c38b80d4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:42.352518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13763-2f83-4ee5-a1e5-d8c38b80d4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:42.352518 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa13763-2f83-4ee5-a1e5-d8c38b80d4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:42.357447 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13763-2f83-4ee5-a1e5-d8c38b80d4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:42.359238 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa13763-2f83-4ee5-a1e5-d8c38b80d4c1 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:42.370236 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1399/2772] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:42.792847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:42.792847 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:42.809081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375effb8-b1e3-4f58-a7e7-caf41f63812e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:42.815153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:42.815921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:42.816203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:42.816473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] PATH_INFO: `/v3/projects/412363788efc485d87acae6c824e0851` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:42.817912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=412363788efc485d87acae6c824e0851)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:42.820139 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:42.921153 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412363788efc485d87acae6c824e0851', 'request_id': 'req-38490fe4-49f2-4941-838f-6d46669cab61'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.927827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412363788efc485d87acae6c824e0851', 'request_id': 'req-38490fe4-49f2-4941-838f-6d46669cab61'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.934514 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412363788efc485d87acae6c824e0851', 'request_id': 'req-38490fe4-49f2-4941-838f-6d46669cab61'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.981864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] The token cache is being invalidate because project 412363788efc485d87acae6c824e0851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:42.982237 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.982901 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-38490fe4-49f2-4941-838f-6d46669cab61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:42.984435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1374/2773] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:42 2022] DELETE /identity/v3/projects/412363788efc485d87acae6c824e0851 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:42.991727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de49244-5e5f-46c8-bf08-e7169613316c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:42.992233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de49244-5e5f-46c8-bf08-e7169613316c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:42.992356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de49244-5e5f-46c8-bf08-e7169613316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.000090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:43.014994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0de49244-5e5f-46c8-bf08-e7169613316c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:43.016184 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0de49244-5e5f-46c8-bf08-e7169613316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.023981 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.024271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.024549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.055955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:43.057075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.077856 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1400/2774] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:43.086747 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.087041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.087364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.106927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:43.108129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.115985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1401/2775] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:43.120190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.120960 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1375/2776] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:43:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:43.129780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8591-66bb-4b99-805c-422d6c0ae14d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.135406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.135785 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.136065 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.136351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.137036 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:43.137338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.139132 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.151701 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1402/2777] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:43:43.156260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.167697 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bbbc1-cc78-406e-a3a0-d3d79c838a0e None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.172262 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.172884 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.173169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.173449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.174080 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.175974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.187854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:43.271519 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.273245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.291949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1376/2778] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:43 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:43.295827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.352425 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8327c2bc-a646-4888-858e-ef2015e6b695 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.365200 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544e10-2166-409b-9a28-2d469bdae0e3 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.372244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.372883 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.373169 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.373439 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.374070 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.376528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.434781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.435390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.435670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.435947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.462916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:43.463470 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1377/2779] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:43 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:43:43.464364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.465972 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.477947 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87f7e2-0b7a-40d7-8521-4be6c04a08ff admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.482721 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.483339 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.483861 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.483969 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] PATH_INFO: `/v3/users/27a74a36c48148e0ba0be713cbe4aa11` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.485059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a74a36c48148e0ba0be713cbe4aa11)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.486987 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.496779 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.498379 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.507129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1403/2780] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:43:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:43:43.507929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.518705 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d30c35-0880-4d55-9e68-95468cf7cc91 service swift] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.523517 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.524126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.524451 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.524722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.525391 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.526969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.543500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1404/2781] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:43.547204 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.557432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217731ad-9f28-4afd-82ba-a4f11e7c3264 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.563432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.563432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.563432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.563432 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.563816 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.565374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.575849 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1405/2782] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:43.579731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.589526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b35f3a-8035-4113-80ad-c2688fd3ecdd admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.594412 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d400-5462-4787-bad7-a02e75bdf312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.594960 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d400-5462-4787-bad7-a02e75bdf312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.595233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d400-5462-4787-bad7-a02e75bdf312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.595502 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d400-5462-4787-bad7-a02e75bdf312 admin admin] PATH_INFO: `/v3/projects/49ae1d33c9874be780d3d55c0d2fc784/users/18d0202c683747c299c3ea857f72c3f5/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.609787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2d400-5462-4787-bad7-a02e75bdf312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ae1d33c9874be780d3d55c0d2fc784, user_id=18d0202c683747c299c3ea857f72c3f5, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.615921 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2d400-5462-4787-bad7-a02e75bdf312 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.679105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a74a36c48148e0ba0be713cbe4aa11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:43.679793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a74a36c48148e0ba0be713cbe4aa11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:43.687028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a74a36c48148e0ba0be713cbe4aa11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:43.687724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a74a36c48148e0ba0be713cbe4aa11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:43.698097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1406/2783] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:43 2022] PUT /identity/v3/projects/49ae1d33c9874be780d3d55c0d2fc784/users/18d0202c683747c299c3ea857f72c3f5/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:43.702986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.703271 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.703560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.723195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:43.725318 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.782499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-d7c9ea95-64af-49f7-8aee-b02194fabea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a74a36c48148e0ba0be713cbe4aa11'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:43.784774 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1407/2784] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:43.795456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.796908 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.804523 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1378/2785] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:43 2022] DELETE /identity/v3/users/27a74a36c48148e0ba0be713cbe4aa11 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:43.808586 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.808943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.809173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.832513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:43.833817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:43.864216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581b313-da50-4433-aba8-334792f37bc2 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.885046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:43.897745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1379/2786] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:43.907484 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:43.907767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:43.924224 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7604426-721d-4986-9c2f-0912260fbc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.924888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7604426-721d-4986-9c2f-0912260fbc6a service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.925185 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7604426-721d-4986-9c2f-0912260fbc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.925461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7604426-721d-4986-9c2f-0912260fbc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.936285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7604426-721d-4986-9c2f-0912260fbc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:43.937945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7604426-721d-4986-9c2f-0912260fbc6a service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:43.949115 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1408/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:43.983338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e1809-d011-4ff4-8b32-8bc86166466a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:43.987432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:43.988235 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:43.988935 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:43.990078 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:43.999634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:44.002836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:44.011582 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1380/2788] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:43:44.372283 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:44.372784 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:44.372930 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:44.393940 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:44.396349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:44.454709 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1409/2789] 10.222.0.42 () {58 vars in 967 bytes} [Tue Oct 4 10:43:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:44.468810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:44.470294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:44.544347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677838c-25b0-463f-bf0a-bccc6fa33b2c service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:44.550054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e34b5-8be3-4c85-93d6-6cfb2844a580 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:44.550649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e34b5-8be3-4c85-93d6-6cfb2844a580 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:44.551029 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:44.551175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e34b5-8be3-4c85-93d6-6cfb2844a580 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:44.551358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e34b5-8be3-4c85-93d6-6cfb2844a580 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:44.553690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:44.569142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e34b5-8be3-4c85-93d6-6cfb2844a580 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:44.570859 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e34b5-8be3-4c85-93d6-6cfb2844a580 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:44.579170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ef606-42df-4543-a74e-22167a744d5d None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:44.582380 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1381/2790] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:43:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:43:44.583848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:44.584676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:44.585162 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:44.585445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] PATH_INFO: `/v3/projects/8995416d93f5422a9ca6213641b93dad` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:44.587322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8995416d93f5422a9ca6213641b93dad)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:44.590408 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:44.765616 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8995416d93f5422a9ca6213641b93dad', 'request_id': 'req-33d19452-1bd0-4586-b44f-b057785f6744'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:44.772576 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8995416d93f5422a9ca6213641b93dad', 'request_id': 'req-33d19452-1bd0-4586-b44f-b057785f6744'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:44.855372 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8995416d93f5422a9ca6213641b93dad', 'request_id': 'req-33d19452-1bd0-4586-b44f-b057785f6744'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:44.866988 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] The token cache is being invalidate because project 8995416d93f5422a9ca6213641b93dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:44.867280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:44.868027 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-33d19452-1bd0-4586-b44f-b057785f6744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:44.869562 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1410/2791] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:44 2022] DELETE /identity/v3/projects/8995416d93f5422a9ca6213641b93dad => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:44.874332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:44.874619 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:44.874891 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:44.895426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:44.897581 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:44.960591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1382/2792] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:44.968704 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:44.969024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:44.969306 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:44.991456 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:44.996063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:45.006457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1411/2793] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:45.011225 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.022706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede9bfd-2eb3-4f36-ad57-269a9bf549ac None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.027607 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.028248 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.028530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.028799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.029480 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:45.029782 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.031436 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.054848 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1383/2794] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:43:45.059690 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.077608 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4250c-b382-4778-b9b9-a828ff54819c None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.084045 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098fbad-4946-4832-aab3-6a39da0b8c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.084998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098fbad-4946-4832-aab3-6a39da0b8c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.085585 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098fbad-4946-4832-aab3-6a39da0b8c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.086021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098fbad-4946-4832-aab3-6a39da0b8c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.087822 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098fbad-4946-4832-aab3-6a39da0b8c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.090364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098fbad-4946-4832-aab3-6a39da0b8c99 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.225377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1412/2795] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:45 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:45.230692 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.246156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29dacd-51db-45a6-af3b-5c12755e9e37 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.255821 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:45.302618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.303211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.303484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.303750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.304372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.306369 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.361476 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.361760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.362038 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.382122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:45.383356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:45.419481 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:45.441543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1413/2796] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:43:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:45.454848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.456421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:45.474761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1384/2797] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:45 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:45.479320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.489224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb42dd-3d2a-40e6-8d50-6e3c5b2b2f9d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.494336 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.495060 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.496073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.496073 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.496732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.503029 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.518995 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1385/2798] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:43:45.519950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.531903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0f939-e80c-4aca-9f47-01053d37b7f1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.537363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.537974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.538311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.538594 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.539420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.540214 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e6d82-f497-4ca0-aba1-8d9723e79aeb None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.541075 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.549641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9167b-3ec7-4edc-b096-e77a1992c832 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.553662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9167b-3ec7-4edc-b096-e77a1992c832 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.553662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9167b-3ec7-4edc-b096-e77a1992c832 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.553662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9167b-3ec7-4edc-b096-e77a1992c832 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.553936 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1386/2799] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:45.557346 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.563260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9167b-3ec7-4edc-b096-e77a1992c832 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.564936 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9167b-3ec7-4edc-b096-e77a1992c832 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.571292 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da05f4e-434f-4310-9b89-a3cb39d5fa62 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.574213 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1414/2800] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:43:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:43:45.577647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.578724 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.579045 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.579345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] PATH_INFO: `/v3/projects/9ebfaa2af25e42b19119e2548e7b5eca/users/3dfba289f8b449e2bada73b344056eb3/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.596793 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebfaa2af25e42b19119e2548e7b5eca, user_id=3dfba289f8b449e2bada73b344056eb3, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.599877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.749195 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1387/2801] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:45 2022] PUT /identity/v3/projects/9ebfaa2af25e42b19119e2548e7b5eca/users/3dfba289f8b449e2bada73b344056eb3/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:45.753825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35360dc-39c3-499b-9621-78f69d1a9c4c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.754552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35360dc-39c3-499b-9621-78f69d1a9c4c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.754552 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35360dc-39c3-499b-9621-78f69d1a9c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.773931 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d35360dc-39c3-499b-9621-78f69d1a9c4c None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:45.775138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d35360dc-39c3-499b-9621-78f69d1a9c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:45.835102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1415/2802] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:45.846549 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:45.848277 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:45.919963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f60933-f663-45f9-8959-8f59f035b328 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:45.924576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:45.925347 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:45.925630 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:45.925993 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:45.943840 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:45.945767 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:45.959709 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1388/2803] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:46.457742 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7bd4-f39b-4744-92a2-d0ccc3d0316c None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:46.458727 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7bd4-f39b-4744-92a2-d0ccc3d0316c None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:46.458818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc7bd4-f39b-4744-92a2-d0ccc3d0316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:46.511699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b0dc7bd4-f39b-4744-92a2-d0ccc3d0316c None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:46.513708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b0dc7bd4-f39b-4744-92a2-d0ccc3d0316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:46.596428 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1416/2804] 10.222.0.42 () {58 vars in 967 bytes} [Tue Oct 4 10:43:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:46.610356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:46.675203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:46.682827 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca8087-a4e0-469b-a84d-5511bfab7a66 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:46.690252 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:46.690923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:46.691927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:46.691927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:46.702495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:46.704333 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:46.716653 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1389/2805] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:43:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:43:47.899311 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:47.900282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:47.900560 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:47.934416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] MFA Rules not processed for user `3dfba289f8b449e2bada73b344056eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:47.936048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:47.945245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:48.012627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1417/2806] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:48.024563 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:48.026095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:48.135649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4a802-431a-4208-be2d-299763a80f1f service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:48.140236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f372d04-c6a6-4605-ae37-c9e2afef71ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:48.140799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372d04-c6a6-4605-ae37-c9e2afef71ac service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:48.141155 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372d04-c6a6-4605-ae37-c9e2afef71ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:48.141375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372d04-c6a6-4605-ae37-c9e2afef71ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:48.155736 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372d04-c6a6-4605-ae37-c9e2afef71ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:48.156031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372d04-c6a6-4605-ae37-c9e2afef71ac service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:48.167975 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1390/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:49.981887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:49.984382 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:50.002069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97497-477c-4218-b8cc-a6155e99887b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:50.008506 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:50.009490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.010151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.010668 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] PATH_INFO: `/v3/users/56bfa1688ede455e9ec01c5b274747c1` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.012233 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=56bfa1688ede455e9ec01c5b274747c1)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:50.014766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:50.255998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1a18-2f58-4edf-8246-049b5fdb9556 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.256597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1a18-2f58-4edf-8246-049b5fdb9556 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.256597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a1a18-2f58-4edf-8246-049b5fdb9556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.262956 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56bfa1688ede455e9ec01c5b274747c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:50.263770 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56bfa1688ede455e9ec01c5b274747c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:50.272985 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56bfa1688ede455e9ec01c5b274747c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:50.273838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56bfa1688ede455e9ec01c5b274747c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:50.295482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-591a1a18-2f58-4edf-8246-049b5fdb9556 None None] MFA Rules not processed for user `18d0202c683747c299c3ea857f72c3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:50.297159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-591a1a18-2f58-4edf-8246-049b5fdb9556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:50.371321 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56bfa1688ede455e9ec01c5b274747c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:50.398718 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1391/2808] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:43:50.400407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e7eb8-7335-4758-a0dd-2e7bab83d4be None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.400810 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e7eb8-7335-4758-a0dd-2e7bab83d4be None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.401517 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e7eb8-7335-4758-a0dd-2e7bab83d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.410763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1418/2809] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:43:49 2022] DELETE /identity/v3/users/56bfa1688ede455e9ec01c5b274747c1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:50.412522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:50.414457 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:50.429665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bb0e7eb8-7335-4758-a0dd-2e7bab83d4be None None] MFA Rules not processed for user `3dfba289f8b449e2bada73b344056eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:50.431706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bb0e7eb8-7335-4758-a0dd-2e7bab83d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:50.493445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015b70c-2c0a-4449-80c1-fa31f8bc2bfc admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:50.501395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:50.503972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.503972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.503972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.503972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:50.506543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:50.507927 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1392/2810] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:50.509992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.510665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.511156 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.532614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:50.542357 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:50.543687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:50.557136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1393/2811] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:50.568665 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:50.573268 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:50.616740 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1419/2812] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:50 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:50.624986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:50.637859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:50.647726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4f55d-ba5c-4ccb-99c7-84b3a6770c7b None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:50.708678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:50.708678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.708678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.708678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.709210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ab7b9-bf28-4d9d-920f-4ace69eedbbb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:50.713824 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:50.714478 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.714825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.715175 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.716057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:50.718025 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:50.718364 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:50.719900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:50.730576 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1394/2813] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:50.829138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1420/2814] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:50 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:50.833950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:50.844540 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178f2af-60d2-4253-ab88-30d0bc4aad84 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:50.849976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:50.850610 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.850887 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.851159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.851803 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:50.853662 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:50.865708 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1395/2815] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:50.871308 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:50.882641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf8d49-e838-4bcc-bacf-3eabf4a82917 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:50.887553 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:50.888234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:50.888538 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:50.888848 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] PATH_INFO: `/v3/projects/0f87a3ef42314ffd897888f28aba24be/users/4c81771b1c9c463e9403c90d64e9ea35/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:50.905741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f87a3ef42314ffd897888f28aba24be, user_id=4c81771b1c9c463e9403c90d64e9ea35, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:50.908438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:50.989903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1421/2816] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:50 2022] PUT /identity/v3/projects/0f87a3ef42314ffd897888f28aba24be/users/4c81771b1c9c463e9403c90d64e9ea35/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:50.994244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:51.007495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473a70c-1c27-4678-a24c-06e3a65422b6 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:51.064163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:51.090699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:51.091417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.092109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.092109 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.092825 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:51.094521 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:51.108924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1396/2817] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:51.109980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:51.124484 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218858ef-6aef-4dcb-b4ab-409c9b15d5ca admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:51.131562 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:51.132375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.132664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.132971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] PATH_INFO: `/v3/projects/0f87a3ef42314ffd897888f28aba24be/users/4c81771b1c9c463e9403c90d64e9ea35/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.134985 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f87a3ef42314ffd897888f28aba24be, user_id=4c81771b1c9c463e9403c90d64e9ea35, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:51.136907 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270a466-838e-45dc-9604-21926f27376c admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:51.140482 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:51.141812 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:51.142651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.143230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.143715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.146444 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:51.157031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1397/2818] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:51 2022] PUT /identity/v3/projects/0f87a3ef42314ffd897888f28aba24be/users/4c81771b1c9c463e9403c90d64e9ea35/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:51.157658 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:51.159137 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e37f68-6dbe-4095-96c3-ea27bd82a1b0 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:51.168346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1422/2819] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:51.301573 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:51.303583 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:51.350082 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:51.400449 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11483c21-d138-4d67-b826-a97cf884d344 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:51.405338 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:51.405886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.406261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.406466 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] PATH_INFO: `/v3/projects/eed55f7ab39c4a60ab366e77e0e1d7a0` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.407618 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed55f7ab39c4a60ab366e77e0e1d7a0)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:51.409568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:51.534113 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed55f7ab39c4a60ab366e77e0e1d7a0', 'request_id': 'req-b1f07df2-78db-4052-9c62-e601a69d61f1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:51.543964 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed55f7ab39c4a60ab366e77e0e1d7a0', 'request_id': 'req-b1f07df2-78db-4052-9c62-e601a69d61f1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:51.553163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed55f7ab39c4a60ab366e77e0e1d7a0', 'request_id': 'req-b1f07df2-78db-4052-9c62-e601a69d61f1'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:51.676119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] The token cache is being invalidate because project eed55f7ab39c4a60ab366e77e0e1d7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:43:51.676119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:51.676119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-b1f07df2-78db-4052-9c62-e601a69d61f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:43:51.677424 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1398/2820] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:43:51 2022] DELETE /identity/v3/projects/eed55f7ab39c4a60ab366e77e0e1d7a0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:43:51.682210 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.682505 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.682787 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.716380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:51.717682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:51.817788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1423/2821] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:51.825384 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.825909 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.826028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.853533 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:51.854764 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:51.863249 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1399/2822] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:51.867430 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:51.878979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90977c4-58b5-4432-8894-2de05c7f50b1 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:51.884352 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:51.885004 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.885288 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.885579 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.886246 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:43:51.886548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:51.888154 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:51.900635 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1424/2823] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:43:51.905307 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:51.916790 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b1f0-a577-4ccd-bd7f-922c6aae50d4 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:51.921687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:51.922312 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:51.922629 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:51.922903 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:51.923541 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:51.925519 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:52.042507 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1400/2824] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:43:51 2022] POST /identity/v3/projects => generated 391 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:43:52.046895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:52.053794 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:52.065005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034cc12-b5a9-4df8-88ed-3097c053675a admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:52.139625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:52.141121 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:52.141662 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:52.142174 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:52.143043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:52.146736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:52.254251 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1425/2825] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:43:52 2022] POST /identity/v3/users => generated 296 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:43:52.258658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:52.272066 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b56877-1239-4210-82a1-977fbf7ef721 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:52.277396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:52.278048 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:52.279170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:52.279170 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:52.280184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:52.282244 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:52.295627 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1401/2826] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:52.300080 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:52.312348 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10295d0f-9e48-4544-9005-a9897b8b9b06 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:52.319205 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef1bbe-dd7d-4d78-9eb4-3b9350837c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:52.320102 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1bbe-dd7d-4d78-9eb4-3b9350837c1a admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:52.320943 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1bbe-dd7d-4d78-9eb4-3b9350837c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:52.321607 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef1bbe-dd7d-4d78-9eb4-3b9350837c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:52.323760 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1bbe-dd7d-4d78-9eb4-3b9350837c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:52.325536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef1bbe-dd7d-4d78-9eb4-3b9350837c1a admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:52.338605 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1426/2827] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:43:52.343359 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:52.354340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ed6f6-a72e-4f8c-8d82-45e37e2ba3d4 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:52.359313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:52.360232 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:52.360722 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:52.361218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] PATH_INFO: `/v3/projects/72d0d5ac4e3b46fb9f167a255a8edea8/users/47bf613056b34e968b04e24693086ad9/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:52.378146 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d0d5ac4e3b46fb9f167a255a8edea8, user_id=47bf613056b34e968b04e24693086ad9, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:52.382183 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:52.463617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1402/2828] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:43:52 2022] PUT /identity/v3/projects/72d0d5ac4e3b46fb9f167a255a8edea8/users/47bf613056b34e968b04e24693086ad9/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:43:52.475119 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98855df2-f900-45b3-93eb-8ad6156d9f15 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:52.475406 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98855df2-f900-45b3-93eb-8ad6156d9f15 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:52.475684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98855df2-f900-45b3-93eb-8ad6156d9f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:52.512890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-98855df2-f900-45b3-93eb-8ad6156d9f15 None None] MFA Rules not processed for user `47bf613056b34e968b04e24693086ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:52.514134 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-98855df2-f900-45b3-93eb-8ad6156d9f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:52.589819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1427/2829] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:52.601601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:52.603431 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:52.678690 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca813b0-71bb-4f36-9887-e87cb1b7f846 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:52.682907 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55965a77a0fd41ce8007cea14d081007', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:52.683602 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:52.683906 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:52.684202 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:52.694497 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:52.696036 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:52.702621 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:52.710715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1403/2830] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:57.985147 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2fa93-fdaf-4f0f-8a01-b30b7e3b94b5 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:57.985788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2fa93-fdaf-4f0f-8a01-b30b7e3b94b5 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:57.985788 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2fa93-fdaf-4f0f-8a01-b30b7e3b94b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:58.014295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-23f2fa93-fdaf-4f0f-8a01-b30b7e3b94b5 None None] MFA Rules not processed for user `4c81771b1c9c463e9403c90d64e9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:43:58.015508 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-23f2fa93-fdaf-4f0f-8a01-b30b7e3b94b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:58.074646 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:43:58.083926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1428/2831] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:43:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:43:58.097959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:43:58.099568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:58.207159 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55035-776a-4860-84b8-6fe88a5ff556 service cinder] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:43:58.234954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:43:58.234954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:43:58.234954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:43:58.234954 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:43:58.235377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:43:58.294382 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:43:58.296017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:43:58.313963 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1404/2832] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:43:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:44:01.571821 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7b0f-7230-4467-8984-240e84f67573 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:01.572301 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7b0f-7230-4467-8984-240e84f67573 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:01.572402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7b0f-7230-4467-8984-240e84f67573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:01.592929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-04aa7b0f-7230-4467-8984-240e84f67573 None None] MFA Rules not processed for user `18d0202c683747c299c3ea857f72c3f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:01.594200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-04aa7b0f-7230-4467-8984-240e84f67573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:01.602319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1429/2833] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:08.301720 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=65 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:11.587618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:13.379044 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:13.438955 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c2bbb-1f47-48a8-bfbf-37e20ce3b0a9 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:13.444057 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:13.444669 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:13.444973 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:13.445246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] PATH_INFO: `/v3/users/47bf613056b34e968b04e24693086ad9` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:13.446340 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bf613056b34e968b04e24693086ad9)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:13.448294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:13.648409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bf613056b34e968b04e24693086ad9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:13.649137 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bf613056b34e968b04e24693086ad9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:13.656555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bf613056b34e968b04e24693086ad9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:13.657228 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bf613056b34e968b04e24693086ad9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:13.770209 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bf613056b34e968b04e24693086ad9'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:13.789233 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1405/2834] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:44:13 2022] DELETE /identity/v3/users/47bf613056b34e968b04e24693086ad9 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:44:13.793713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:13.794211 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:13.794313 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:13.813803 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:13.815024 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:13.882009 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1430/2835] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:13.890506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:13.892102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:13.950054 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2839a-95a6-4d26-bcf1-1f80eb7a102c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:13.953808 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cbc36-640a-4178-9870-0dc8e1d6add6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:13.954433 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cbc36-640a-4178-9870-0dc8e1d6add6 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:13.954844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cbc36-640a-4178-9870-0dc8e1d6add6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:13.955122 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cbc36-640a-4178-9870-0dc8e1d6add6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:13.965145 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cbc36-640a-4178-9870-0dc8e1d6add6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:13.966578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cbc36-640a-4178-9870-0dc8e1d6add6 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:13.976391 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1406/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:44:14.486079 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:14.487522 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:14.499817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad845874-5657-48c0-81b2-e7677ae27b9b None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:14.504473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:14.505685 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:14.506666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:14.506666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] PATH_INFO: `/v3/projects/72d0d5ac4e3b46fb9f167a255a8edea8` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:14.507602 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d0d5ac4e3b46fb9f167a255a8edea8)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:14.509575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:14.611148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d0d5ac4e3b46fb9f167a255a8edea8', 'request_id': 'req-757061f9-81b7-418d-8226-f618e4c4ffce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:14.617429 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d0d5ac4e3b46fb9f167a255a8edea8', 'request_id': 'req-757061f9-81b7-418d-8226-f618e4c4ffce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:14.702461 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d0d5ac4e3b46fb9f167a255a8edea8', 'request_id': 'req-757061f9-81b7-418d-8226-f618e4c4ffce'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:14.714846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] The token cache is being invalidate because project 72d0d5ac4e3b46fb9f167a255a8edea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:44:14.714846 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:14.715376 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-757061f9-81b7-418d-8226-f618e4c4ffce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:44:14.716884 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1431/2837] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:44:14 2022] DELETE /identity/v3/projects/72d0d5ac4e3b46fb9f167a255a8edea8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:44:14.721854 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:14.722328 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:14.722451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:14.753119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:14.754320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:14.772005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:14.813350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1407/2838] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:44:14.822019 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:14.822548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:14.822548 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:14.844473 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:14.845719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:14.853731 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1432/2839] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:14.858019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:14.869281 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3d43e-3d71-464f-a3be-e60729754e13 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:14.874494 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:14.875096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:14.879511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:14.879511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:14.879511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:44:14.879511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:14.879511 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:14.891605 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1408/2840] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:44:14.896500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:14.905693 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2042c-28ed-4106-a24d-5e9fc35f0807 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:14.910349 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:14.910945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:14.911227 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:14.911947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:14.912599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:14.914526 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:15.004367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1433/2841] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:44:14 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:44:15.009034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:15.019108 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09a0a4-2664-4d99-990d-ddfda060782e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:15.074301 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:15.074898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.075176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.075445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.076090 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:15.078056 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:15.193699 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1409/2842] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:44:15 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:44:15.200927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:15.211044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c65d74-de56-4c0c-9504-15ac58f1e418 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:15.215798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:15.216384 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.216659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.216958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.217590 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:15.219178 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:15.229282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1434/2843] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:44:15.233069 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:15.244085 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b4a97-d597-4717-8d13-285f369856c1 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:15.248779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff874f-ad00-4a28-bd7c-f4748f1c3af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:15.249405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff874f-ad00-4a28-bd7c-f4748f1c3af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.249681 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff874f-ad00-4a28-bd7c-f4748f1c3af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.249950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff874f-ad00-4a28-bd7c-f4748f1c3af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.250612 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff874f-ad00-4a28-bd7c-f4748f1c3af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:15.252178 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff874f-ad00-4a28-bd7c-f4748f1c3af8 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:15.264674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1410/2844] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:44:15.268919 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:15.279338 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8df89-e2c7-4366-a5e3-d9a342c25a27 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:15.283929 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:15.284574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.284853 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.285165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] PATH_INFO: `/v3/projects/9171bd2ad8cc4736bb8c585f40040795/users/f5073f864387435281b00e532bf236d0/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.300477 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9171bd2ad8cc4736bb8c585f40040795, user_id=f5073f864387435281b00e532bf236d0, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:15.304319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:15.372520 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1435/2845] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:44:15 2022] PUT /identity/v3/projects/9171bd2ad8cc4736bb8c585f40040795/users/f5073f864387435281b00e532bf236d0/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:44:15.377766 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f3e0d-0ce5-4720-935e-b953f04dd587 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.378197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f3e0d-0ce5-4720-935e-b953f04dd587 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.378542 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f3e0d-0ce5-4720-935e-b953f04dd587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.397939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-e66f3e0d-0ce5-4720-935e-b953f04dd587 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:15.399287 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-e66f3e0d-0ce5-4720-935e-b953f04dd587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:15.461260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1411/2846] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:15.471103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:15.474525 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:15.497556 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:15.532114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187a79d-7007-46f9-a9dd-87a369e4150f admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:15.535627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585f907-ad85-4c3a-bbc8-b70d2024bea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:15.536222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585f907-ad85-4c3a-bbc8-b70d2024bea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.537123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585f907-ad85-4c3a-bbc8-b70d2024bea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.537123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585f907-ad85-4c3a-bbc8-b70d2024bea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.546498 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585f907-ad85-4c3a-bbc8-b70d2024bea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:15.547916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585f907-ad85-4c3a-bbc8-b70d2024bea1 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:15.557282 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1436/2847] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:44:15.792027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:15.792027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:15.792027 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:15.808970 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] MFA Rules not processed for user `3dfba289f8b449e2bada73b344056eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:15.810171 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:15.834224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:15.866654 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1412/2848] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:22.245231 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:22.245707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:22.245707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:22.322656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] MFA Rules not processed for user `f5073f864387435281b00e532bf236d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:22.323964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:22.416838 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1437/2849] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:22.421405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:22.435991 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefa043-ac3e-4bad-b635-bad974fddffe None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:22.440994 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:22.441942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:22.442455 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:22.442792 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:22.444193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:22.445505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:22.569320 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1413/2850] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:44:22 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:44:22.574883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:22.587177 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b57dd-3fee-465a-85ca-b3abcf48e35e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:22.660366 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:22.661035 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:22.661319 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:22.661600 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:22.662234 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:22.664187 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:22.765545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1438/2851] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:44:22 2022] POST /identity/v3/users => generated 299 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:44:22.769136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:22.780452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03ce8d-ed96-4ff6-a462-3f4e8cf2ad0e admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:22.785473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:22.786572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:22.786966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:22.787426 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:22.788276 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:22.789912 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:22.804246 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1414/2852] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:44:22.805110 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:22.817874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9305b2-e1b6-4781-911c-6853dd24e770 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:22.823659 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:22.824106 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:22.824392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:22.824672 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] PATH_INFO: `/v3/projects/3bf29295fcd14293848aec5a9be10e4b/users/d17edc4bf634456e84f399ce2bf03496/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:22.852122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf29295fcd14293848aec5a9be10e4b, user_id=d17edc4bf634456e84f399ce2bf03496, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:22.854916 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:22.959778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1439/2853] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:44:22 2022] PUT /identity/v3/projects/3bf29295fcd14293848aec5a9be10e4b/users/d17edc4bf634456e84f399ce2bf03496/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:44:22.963885 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:22.974327 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b2c02-8aa2-4da3-bc7c-4cfeb145d3bd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:23.038102 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14ec19-11ac-4886-a8b8-013217a0ce73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:23.038715 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14ec19-11ac-4886-a8b8-013217a0ce73 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:23.038998 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14ec19-11ac-4886-a8b8-013217a0ce73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:23.039272 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14ec19-11ac-4886-a8b8-013217a0ce73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:23.039943 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14ec19-11ac-4886-a8b8-013217a0ce73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:23.041580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14ec19-11ac-4886-a8b8-013217a0ce73 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:23.051565 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1415/2854] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:44:23.055994 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:23.066636 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576775d3-cc7e-475b-973e-43cd49a4e712 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:23.072068 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:23.072721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:23.073057 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:23.073367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] PATH_INFO: `/v3/projects/3bf29295fcd14293848aec5a9be10e4b/users/d17edc4bf634456e84f399ce2bf03496/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:23.075272 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf29295fcd14293848aec5a9be10e4b, user_id=d17edc4bf634456e84f399ce2bf03496, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:23.078003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:23.090201 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1440/2855] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:44:23 2022] PUT /identity/v3/projects/3bf29295fcd14293848aec5a9be10e4b/users/d17edc4bf634456e84f399ce2bf03496/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:44:29.941149 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5289-76bb-49a0-afab-b50c0078f744 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:29.941685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5289-76bb-49a0-afab-b50c0078f744 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:29.941685 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5289-76bb-49a0-afab-b50c0078f744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:29.975984 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-5d1a5289-76bb-49a0-afab-b50c0078f744 None None] MFA Rules not processed for user `d17edc4bf634456e84f399ce2bf03496`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:29.977502 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5d1a5289-76bb-49a0-afab-b50c0078f744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:30.075924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1416/2856] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:30.091669 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:30.093420 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:30.101676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:30.157550 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317b58b-d5be-4491-8f6a-6bcc64bbe0df admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:30.162063 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:30.162775 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:30.163947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:30.163947 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:30.232653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:30.234170 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:30.244135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1441/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:44:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:44:30.634014 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:30.634306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:30.637084 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:30.654760 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:30.655967 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:30.673750 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:30.711435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1417/2858] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:44:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:44:30.725138 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:30.726656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:30.794922 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7329-1721-4034-b26e-c71a3414bbcf service glance] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:30.799920 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82870a6-d386-4011-aab2-a4ff2e70d405 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:30.800521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82870a6-d386-4011-aab2-a4ff2e70d405 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:30.800795 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82870a6-d386-4011-aab2-a4ff2e70d405 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:30.801103 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82870a6-d386-4011-aab2-a4ff2e70d405 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:30.809155 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82870a6-d386-4011-aab2-a4ff2e70d405 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:30.810724 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82870a6-d386-4011-aab2-a4ff2e70d405 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:30.819240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1442/2859] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:44:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:44:33.236622 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:33.239184 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:33.333105 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f42ba-2e97-459c-a24b-f60bc0001491 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:33.345580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:33.345580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:33.345580 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:33.346091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:33.349294 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:33.376308 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:33.380957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:33.408723 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1418/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:44:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:44:40.976707 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=78 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:41.102061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21925078-cc61-4c8d-b9cf-07b0aecfbd20 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:41.102346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21925078-cc61-4c8d-b9cf-07b0aecfbd20 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:41.102631 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21925078-cc61-4c8d-b9cf-07b0aecfbd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:41.125828 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-21925078-cc61-4c8d-b9cf-07b0aecfbd20 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:41.127005 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-21925078-cc61-4c8d-b9cf-07b0aecfbd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:41.134541 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1443/2861] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:44:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:41.155161 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:41.169241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dc9e5-858c-4ebe-a673-c6872eb577d0 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:41.174930 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:41.175652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:41.176031 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:41.176413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:41.187435 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:41.189270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:41.198775 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1419/2862] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:44:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:44:42.950212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5990b94-c8fc-43f9-b227-31f2902511a2 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:42.950854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5990b94-c8fc-43f9-b227-31f2902511a2 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:42.950854 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5990b94-c8fc-43f9-b227-31f2902511a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:42.979421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-b5990b94-c8fc-43f9-b227-31f2902511a2 None None] MFA Rules not processed for user `f5073f864387435281b00e532bf236d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:42.980719 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b5990b94-c8fc-43f9-b227-31f2902511a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:42.990360 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1444/2863] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:44:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:44:45.447331 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:45.449498 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:45.521396 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37503c6d-4163-4925-95d6-bac9bc6fd73e service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:45.525218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:45.525850 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:45.526286 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:45.526473 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:45.534175 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:45.604989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:45.626741 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:45.633376 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:45.642158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1420/2864] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:44:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:44:47.757336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2d862-95c0-4899-bfe2-bee3330ec37e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:47.757892 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2d862-95c0-4899-bfe2-bee3330ec37e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:47.757998 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2d862-95c0-4899-bfe2-bee3330ec37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:47.802772 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-bbf2d862-95c0-4899-bfe2-bee3330ec37e None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:44:47.804016 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-bbf2d862-95c0-4899-bfe2-bee3330ec37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:44:47.816083 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1445/2865] 10.222.0.42 () {58 vars in 1009 bytes} [Tue Oct 4 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:44:47.839318 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:47.853898 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336f5e3-a6cb-45d3-9884-b4d50459af55 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:47.860701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:47.861352 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:47.861634 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:47.861913 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:47.873855 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:47.876065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:47.888649 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1421/2866] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:44:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:44:57.796294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=9 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:57.883019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=11 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:01.017449 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:01.017969 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:01.018090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:01.037418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:45:01.038610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:01.045654 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1446/2867] 10.222.0.42 () {58 vars in 967 bytes} [Tue Oct 4 10:45:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:45:01.057853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:01.069893 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a655afa-b40d-46a6-bd10-bbc94d167935 service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:01.074816 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:01.075411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:01.075687 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:01.075957 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:01.084774 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:01.086367 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:01.095099 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1422/2868] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:45:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:45:04.547495 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:04.548979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:04.621732 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daaf19-dfde-4a47-91e6-82090e822949 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:04.626436 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:04.627072 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:04.627351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:04.627629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] PATH_INFO: `/v3/users/3dfba289f8b449e2bada73b344056eb3` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:04.628799 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfba289f8b449e2bada73b344056eb3)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:04.631555 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:04.857769 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfba289f8b449e2bada73b344056eb3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:04.858418 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfba289f8b449e2bada73b344056eb3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:04.865651 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfba289f8b449e2bada73b344056eb3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:04.866245 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfba289f8b449e2bada73b344056eb3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:04.947129 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfba289f8b449e2bada73b344056eb3'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:04.966954 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1447/2869] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:45:04 2022] DELETE /identity/v3/users/3dfba289f8b449e2bada73b344056eb3 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:45:04.967316 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:04.977208 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35365ed4-719b-4f40-9c3a-9dac65023e9a service swift] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:05.030530 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:05.031420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:05.034527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:05.034527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:05.034527 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:05.118725 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:05.120551 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad321-862f-427b-a8ea-3a500e8ad513 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:05.132236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1423/2870] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:45:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:45:05.294385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:05.295825 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:05.367678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca7af8-b28e-4332-9918-7eb7d2c19848 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:05.372741 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:05.373407 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:05.373687 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:05.373962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] PATH_INFO: `/v3/projects/9ebfaa2af25e42b19119e2548e7b5eca` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:05.375118 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebfaa2af25e42b19119e2548e7b5eca)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:05.377091 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:05.393253 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:05.517486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebfaa2af25e42b19119e2548e7b5eca', 'request_id': 'req-def41dfe-8a34-4c67-869c-8c014d6887ff'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:05.523554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebfaa2af25e42b19119e2548e7b5eca', 'request_id': 'req-def41dfe-8a34-4c67-869c-8c014d6887ff'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:05.615656 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebfaa2af25e42b19119e2548e7b5eca', 'request_id': 'req-def41dfe-8a34-4c67-869c-8c014d6887ff'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:05.627033 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] The token cache is being invalidate because project 9ebfaa2af25e42b19119e2548e7b5eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:45:05.627242 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:05.627880 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-def41dfe-8a34-4c67-869c-8c014d6887ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:05.629641 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1448/2871] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:45:05 2022] DELETE /identity/v3/projects/9ebfaa2af25e42b19119e2548e7b5eca => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:45:05.634809 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:05.635415 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:05.635700 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:05.655714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:45:05.656952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:05.718104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1424/2872] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:45:05.728011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:05.728289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:05.728565 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:05.749001 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:45:05.750215 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:05.758570 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1449/2873] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:45:05.762843 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:05.773000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf17a-a2b9-48ef-b381-adf0559431c7 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:05.777578 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:05.778174 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:05.778453 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:05.778726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] PATH_INFO: `/v3/domains` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:05.779363 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81561) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:45:05.779664 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:05.781297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:05.793086 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1425/2874] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:45:05.797259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:05.809581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef043483-4c59-4b00-85af-0c4e84ae68f8 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:05.814363 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:05.814961 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:05.815239 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:05.815511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:05.816144 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:05.818133 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:05.937259 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1450/2875] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:45:05 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:45:05.941926 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:05.952924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3fdc9-7d1c-4527-b49e-59e5428d0248 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:05.969005 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:06.024587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:06.025238 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.025609 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.025881 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.026515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:06.028429 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:06.143129 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1426/2876] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:45:05 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:45:06.148454 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:06.162023 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a78f1-a8a6-4021-ace1-4be93b7268d0 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:06.166883 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:06.167487 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.167766 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.168041 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.168682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:06.170421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:06.184171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1451/2877] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:45:06.188411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:06.199358 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a8741-1a05-449b-b941-582367bc6015 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:06.205203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:06.205888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.206176 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.206447 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.207845 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:06.209506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:06.220928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1427/2878] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:45:06.225990 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:06.239530 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e537-86b3-4e81-992d-7141e37ef987 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:06.244968 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dba0f-dc4b-4f99-9be1-39e3b0f6decf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:06.245627 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dba0f-dc4b-4f99-9be1-39e3b0f6decf admin admin] REQUEST_METHOD: `PUT` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.245927 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dba0f-dc4b-4f99-9be1-39e3b0f6decf admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.246200 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dba0f-dc4b-4f99-9be1-39e3b0f6decf admin admin] PATH_INFO: `/v3/projects/c891e0ad2ca24ede80c8143bfbb06edb/users/1f66f3755cfc4c7fb934a0aaa3c846f8/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.263896 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dba0f-dc4b-4f99-9be1-39e3b0f6decf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c891e0ad2ca24ede80c8143bfbb06edb, user_id=1f66f3755cfc4c7fb934a0aaa3c846f8, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:06.266578 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dba0f-dc4b-4f99-9be1-39e3b0f6decf admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:06.319980 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1452/2879] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:45:06 2022] PUT /identity/v3/projects/c891e0ad2ca24ede80c8143bfbb06edb/users/1f66f3755cfc4c7fb934a0aaa3c846f8/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:45:06.326299 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:06.328406 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:06.331017 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.331390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.332500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.367546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] MFA Rules not processed for user `1f66f3755cfc4c7fb934a0aaa3c846f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:45:06.368764 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:06.412587 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb329b-2a52-431c-affe-142c5d46ce98 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:06.446123 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1453/2880] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:45:06.460112 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:06.461831 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:06.475096 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:06.475710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.475990 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.476261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.478100 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:06.489475 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:06.531733 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e601e-acc1-4a89-9a54-c0617959616e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:06.537754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28785166e8994ba18959790e0e5fd3d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:06.538383 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:06.538767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:06.539044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:06.550048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:06.552529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:06.560133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:06.563859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1454/2881] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:45:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:45:06.579094 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:06.580666 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:06.595597 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1428/2882] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:45:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:45:16.740882 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=17 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:16.838297 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=4 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:19.008490 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:19.018097 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b57c29-b9d6-40d2-bbcc-cc8bc3758471 service glance] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:19.027619 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:19.028315 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:19.028666 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:19.029054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:19.030992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:19.095188 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:19.096629 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0d2cb-ba8f-47ec-8f7f-248b068495a0 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:19.110521 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1455/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:45:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:45:29.102592 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=30 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:37.295345 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:37.364802 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e9d91-018a-4e5f-a8a6-43659b752b48 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:37.369410 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:37.370037 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:37.370317 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:37.370589 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] PATH_INFO: `/v3/users/1f66f3755cfc4c7fb934a0aaa3c846f8` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:37.371714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f66f3755cfc4c7fb934a0aaa3c846f8)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:37.373677 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:37.594761 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f66f3755cfc4c7fb934a0aaa3c846f8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:37.595407 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f66f3755cfc4c7fb934a0aaa3c846f8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:37.603223 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f66f3755cfc4c7fb934a0aaa3c846f8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:37.603924 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f66f3755cfc4c7fb934a0aaa3c846f8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:37.691372 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f66f3755cfc4c7fb934a0aaa3c846f8'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:37.710614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1429/2884] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:45:37 2022] DELETE /identity/v3/users/1f66f3755cfc4c7fb934a0aaa3c846f8 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:45:37.715254 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:37.715978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:37.715978 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:37.734818 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:45:37.740909 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:37.795164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1456/2885] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:45:37.805877 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:37.807341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:37.865190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02cd858-f3bb-4c6b-a89e-f714599d682f admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:37.868658 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:37.869283 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:37.869560 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:37.869830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:37.879341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:37.880757 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:37.890050 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1430/2886] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:45:38.301431 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:38.302874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:38.317527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396432d8-c4d8-492a-8e9e-f4e4ba0909b3 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:38.323235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:38.325393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:38.325393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:38.325393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] PATH_INFO: `/v3/projects/c891e0ad2ca24ede80c8143bfbb06edb` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:38.326354 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c891e0ad2ca24ede80c8143bfbb06edb)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:38.327639 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:38.459932 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c891e0ad2ca24ede80c8143bfbb06edb', 'request_id': 'req-e5d71003-04a5-4ef9-8b12-e99e72238ccb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:38.466290 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c891e0ad2ca24ede80c8143bfbb06edb', 'request_id': 'req-e5d71003-04a5-4ef9-8b12-e99e72238ccb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:38.556544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c891e0ad2ca24ede80c8143bfbb06edb', 'request_id': 'req-e5d71003-04a5-4ef9-8b12-e99e72238ccb'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:38.567486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] The token cache is being invalidate because project c891e0ad2ca24ede80c8143bfbb06edb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:45:38.567773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:38.568389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:45:38.569886 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1457/2887] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:45:38 2022] DELETE /identity/v3/projects/c891e0ad2ca24ede80c8143bfbb06edb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:45:47.885952 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:48.564297 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:45:51.151143 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:45:51.159880 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2158ce-a56f-491e-bb62-dfacb2bf76ec service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:45:51.210779 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:45:51.211489 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:45:51.211771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:45:51.212043 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:45:51.214065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:45:51.223314 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:45:51.224830 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68553fef-f553-4f48-a8c7-608f02cd31e6 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:45:51.233313 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1431/2888] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:46:01.229688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=26 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:46:50.065963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:46:50.067594 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:46:50.143011 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d71003-04a5-4ef9-8b12-e99e72238ccb admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:46:50.148575 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:46:50.149721 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:46:50.150089 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:46:50.150362 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:46:50.152678 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:46:50.166190 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:46:50.169715 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:46:50.179653 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1458/2889] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:46:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:46:52.383501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:46:52.387858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:46:52.387858 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:46:52.433799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] MFA Rules not processed for user `64622f7e541046d69f64a4393316abd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:46:52.435158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:46:52.498501 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1432/2890] 10.222.0.42 () {58 vars in 967 bytes} [Tue Oct 4 10:46:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:46:52.627620 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:46:52.629069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:46:52.719773 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9772c7-6906-4ad5-a179-649462ccdd6b service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:46:52.725020 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e3ba7be5b48443697ad07dc4ffb04fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:46:52.725626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:46:52.725906 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:46:52.726181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:46:52.734260 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:46:52.735833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:46:52.743642 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1459/2891] 10.222.0.42 () {60 vars in 1384 bytes} [Tue Oct 4 10:46:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:46:57.426065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:46:57.427477 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:46:57.500966 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300720f-9873-48a5-a684-1b9d1a1b8fc2 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:46:57.505942 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:46:57.506543 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:46:57.506820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:46:57.507092 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:46:57.508818 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:46:57.521490 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:46:57.523163 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:46:57.534465 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1433/2892] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:46:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:47:01.416037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:01.417817 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:01.484206 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=100 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:01.495159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def2351-0728-4fea-adfd-8e783a919836 service swift] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:01.495159 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:01.495401 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:01.495644 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:01.495915 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] PATH_INFO: `/v3/users/f5073f864387435281b00e532bf236d0` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:01.497163 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5073f864387435281b00e532bf236d0)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:01.499069 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:01.731572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5073f864387435281b00e532bf236d0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:01.732450 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5073f864387435281b00e532bf236d0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:01.739890 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5073f864387435281b00e532bf236d0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:01.740547 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5073f864387435281b00e532bf236d0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:01.807767 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5073f864387435281b00e532bf236d0'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:01.827403 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1460/2893] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:47:01 2022] DELETE /identity/v3/users/f5073f864387435281b00e532bf236d0 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:47:02.070864 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:02.091227 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990ac90-3500-4cbe-b7a5-1ff91c3b18fc service placement] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:02.152980 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:02.153617 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:02.153900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:02.154173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] PATH_INFO: `/v3/projects/9171bd2ad8cc4736bb8c585f40040795` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:02.155438 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9171bd2ad8cc4736bb8c585f40040795)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:02.157417 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:02.160019 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=100 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:02.310261 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9171bd2ad8cc4736bb8c585f40040795', 'request_id': 'req-3c606816-bbd1-40ac-a2e3-207c088b55cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.317403 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9171bd2ad8cc4736bb8c585f40040795', 'request_id': 'req-3c606816-bbd1-40ac-a2e3-207c088b55cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.323548 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9171bd2ad8cc4736bb8c585f40040795', 'request_id': 'req-3c606816-bbd1-40ac-a2e3-207c088b55cb'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.407974 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] The token cache is being invalidate because project 9171bd2ad8cc4736bb8c585f40040795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:47:02.408271 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.408996 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.410680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1434/2894] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:47:02 2022] DELETE /identity/v3/projects/9171bd2ad8cc4736bb8c585f40040795 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:47:02.414569 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:02.416074 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:02.429869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ba222-3bbf-4063-9bb3-ce82fc01b965 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:02.486972 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:02.487613 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:02.487893 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:02.488186 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] PATH_INFO: `/v3/users/d17edc4bf634456e84f399ce2bf03496` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:02.489356 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d17edc4bf634456e84f399ce2bf03496)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:02.491273 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:02.674625 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd17edc4bf634456e84f399ce2bf03496'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.675511 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd17edc4bf634456e84f399ce2bf03496'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.682917 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd17edc4bf634456e84f399ce2bf03496'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.683626 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd17edc4bf634456e84f399ce2bf03496'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.756332 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-374110af-5fc8-4367-af42-7847e5fc06d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd17edc4bf634456e84f399ce2bf03496'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:02.767587 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:02.775385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1461/2895] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:47:02 2022] DELETE /identity/v3/users/d17edc4bf634456e84f399ce2bf03496 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:47:03.048131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.048131 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:03.060836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606816-bbd1-40ac-a2e3-207c088b55cb admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.122688 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.123337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.123462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.123726 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] PATH_INFO: `/v3/projects/3bf29295fcd14293848aec5a9be10e4b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.124826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf29295fcd14293848aec5a9be10e4b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.126744 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:03.249133 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf29295fcd14293848aec5a9be10e4b', 'request_id': 'req-e845e647-339e-490a-b0c8-7e60e21d7161'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:03.255392 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf29295fcd14293848aec5a9be10e4b', 'request_id': 'req-e845e647-339e-490a-b0c8-7e60e21d7161'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:03.260989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf29295fcd14293848aec5a9be10e4b', 'request_id': 'req-e845e647-339e-490a-b0c8-7e60e21d7161'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:03.362177 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] The token cache is being invalidate because project 3bf29295fcd14293848aec5a9be10e4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81561) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:47:03.362467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:03.363091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-e845e647-339e-490a-b0c8-7e60e21d7161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:47:03.364569 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1435/2896] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:47:03 2022] DELETE /identity/v3/projects/3bf29295fcd14293848aec5a9be10e4b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:47:03.369674 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.369962 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.370244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.390021 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:47:03.391165 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:03.458054 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1462/2897] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:47:03.466188 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.466475 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.466753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.486451 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:47:03.487659 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:03.495468 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1436/2898] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:47:03.500160 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.509845 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114127-de16-4855-a63c-ea00c65aa506 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.515385 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.516013 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.516295 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.516573 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] PATH_INFO: `/v3/domains` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.517262 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81560) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:47:03.517566 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.519377 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:03.531958 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1463/2899] 10.222.0.42 () {58 vars in 1153 bytes} [Tue Oct 4 10:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:47:03.536241 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.546939 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f362-2e68-4de8-aa10-4767edb68452 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.551949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.552515 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.552791 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.553089 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.553728 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.555614 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:03.571749 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:03.667869 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1437/2900] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:47:03 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:47:03.671979 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.681874 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad60cbe-a85e-422c-89dd-86c1a7b7f002 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.738471 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.739101 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.739369 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.739667 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] PATH_INFO: `/v3/users` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.740280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.742243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:03.836877 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1464/2901] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:47:03 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:47:03.841140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.851950 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4920d2-95c1-4b31-893f-6139ca83d453 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.856706 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.857337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.857616 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.857888 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] PATH_INFO: `/v3/roles` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.858528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.860251 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:03.870890 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1438/2902] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:47:03.874888 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.884869 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325e04e-a98d-4f99-ac68-0d015f762526 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.890897 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4405e8-9985-4389-9df5-2fdd89c41b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.893902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4405e8-9985-4389-9df5-2fdd89c41b17 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.893902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4405e8-9985-4389-9df5-2fdd89c41b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.893902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4405e8-9985-4389-9df5-2fdd89c41b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.893902 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4405e8-9985-4389-9df5-2fdd89c41b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.894294 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4405e8-9985-4389-9df5-2fdd89c41b17 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:03.906336 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1465/2903] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:47:03.910591 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:03.921067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4fc81e-1954-4161-b326-5aaefb7a169b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:03.925756 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:03.926445 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:03.926727 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:03.927000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] PATH_INFO: `/v3/projects/90a1d65daca849b88e1054b8ea48e0e0/users/807807d370a94aa6a312e51c7a8a2db2/roles/76c32cb614c145319ad750282d6d3551` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:03.942716 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a1d65daca849b88e1054b8ea48e0e0, user_id=807807d370a94aa6a312e51c7a8a2db2, role_id=76c32cb614c145319ad750282d6d3551)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:03.945420 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:04.020457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1439/2904] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:47:03 2022] PUT /identity/v3/projects/90a1d65daca849b88e1054b8ea48e0e0/users/807807d370a94aa6a312e51c7a8a2db2/roles/76c32cb614c145319ad750282d6d3551 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:47:04.025652 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:04.025945 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:04.026226 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:04.045527 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] MFA Rules not processed for user `105ad03a7a5c40c78500f0ee01583fae`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:47:04.046827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:04.100583 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:04.110164 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1466/2905] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:47:04.122212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:04.122212 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:04.178787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a158693-baf9-4bad-9ef9-d390b136db1d admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:04.182408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:04.183017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:04.183332 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:04.184141 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:04.193083 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:04.194555 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:04.205601 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1440/2906] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:47:05.251984 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:05.262827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53abc-8f79-4ab5-a1d4-ca717938c850 None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:05.267492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:05.268113 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:05.268392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:05.268676 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:05.270580 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:05.282402 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:05.283778 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:05.293114 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1467/2907] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:47:05.990593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:06.004218 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:06.005876 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75af344-c5fa-4597-a3a1-f04cb6a03062 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:06.015572 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:06.016595 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:06.017166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:06.017745 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:06.020017 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:06.031691 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:06.033492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:06.055796 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1441/2908] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:47:07.130232 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:07.141963 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854eed8-92e9-4741-be04-700c2cb0477d service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:07.146331 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:07.146957 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:07.147353 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:07.147546 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:07.149683 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:07.214119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:07.215734 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:07.259708 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:07.261617 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:07.274839 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1468/2909] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:47:07.352941 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:07.358065 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066c871-19c4-4f10-9e3a-4d4575477e36 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:07.358208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:07.368356 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:07.369395 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:07.370140 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:07.370886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:07.373768 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:07.391047 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:07.400568 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b09c6-c254-4d70-b718-3e52f8f2ec2a service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:07.424820 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1442/2910] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:47:07.445148 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0870358-4ce6-4319-97b5-1467bbb65dff service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:07.453682 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:07.454843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:07.455395 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:07.455512 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:07.458222 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:07.471944 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:07.481135 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:07.487870 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1469/2911] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:47:10.437856 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:10.438390 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:10.438547 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:10.474211 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] MFA Rules not processed for user `807807d370a94aa6a312e51c7a8a2db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:47:10.478670 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:10.500067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:10.566104 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1443/2912] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:47:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:47:10.570235 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:10.584599 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38059e-6205-44d8-bd6d-4b71cf934055 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:10.590581 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:10.591212 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:10.592276 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:10.592586 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] PATH_INFO: `/v3/projects` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:10.593252 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:10.601912 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:10.725086 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1470/2913] 10.222.0.42 () {60 vars in 1153 bytes} [Tue Oct 4 10:47:10 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:47:10.729350 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:10.740413 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082697d1-f917-44a0-8415-2ead72ff9975 None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:10.802193 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:10.802732 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:10.803142 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:10.803442 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] PATH_INFO: `/v3/users` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:10.804072 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:10.806001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:10.909373 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1444/2914] 10.222.0.42 () {60 vars in 1144 bytes} [Tue Oct 4 10:47:10 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:47:10.913596 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:10.924745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de280c9-cff6-4ff8-9b15-fb2151fb255b admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:10.929610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:10.930216 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:10.930503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:10.931421 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:10.931531 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:10.933225 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:10.948410 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1471/2915] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:47:10.949655 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:10.960401 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe25689-9fed-4fdc-a68d-7cf3341e6802 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:10.965528 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:10.966799 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:10.967136 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:10.967386 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] PATH_INFO: `/v3/projects/8875fee8cc474f3d89ba073ffc19053e/users/a2fbe4aab80c483b9fbfc86ff81daa38/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:10.983705 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8875fee8cc474f3d89ba073ffc19053e, user_id=a2fbe4aab80c483b9fbfc86ff81daa38, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:10.986434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:11.076949 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1445/2916] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:47:10 2022] PUT /identity/v3/projects/8875fee8cc474f3d89ba073ffc19053e/users/a2fbe4aab80c483b9fbfc86ff81daa38/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:47:11.080670 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:11.092415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012a31b-b8b8-4554-97dc-dd8fcd76c823 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:11.120266 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:11.151145 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5c071-eb44-444d-8d0e-701371b29b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:11.151763 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5c071-eb44-444d-8d0e-701371b29b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:11.152044 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5c071-eb44-444d-8d0e-701371b29b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:11.152316 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5c071-eb44-444d-8d0e-701371b29b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:11.152993 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5c071-eb44-444d-8d0e-701371b29b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:11.155833 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5c071-eb44-444d-8d0e-701371b29b3f admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:11.166275 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1472/2917] 10.222.0.42 () {58 vars in 1122 bytes} [Tue Oct 4 10:47:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:47:11.171360 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:11.182087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ec4fa-2475-41bf-99f3-4cc6ebfe2a6c admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:11.186992 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:11.187647 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:11.187928 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:11.188229 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] PATH_INFO: `/v3/projects/8875fee8cc474f3d89ba073ffc19053e/users/a2fbe4aab80c483b9fbfc86ff81daa38/roles/9211b6839ec4401ca9aada37827b6b8b` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:11.190393 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8875fee8cc474f3d89ba073ffc19053e, user_id=a2fbe4aab80c483b9fbfc86ff81daa38, role_id=9211b6839ec4401ca9aada37827b6b8b)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:11.193138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:11.204826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1446/2918] 10.222.0.42 () {60 vars in 1483 bytes} [Tue Oct 4 10:47:11 2022] PUT /identity/v3/projects/8875fee8cc474f3d89ba073ffc19053e/users/a2fbe4aab80c483b9fbfc86ff81daa38/roles/9211b6839ec4401ca9aada37827b6b8b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:47:17.288416 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925a080-6a7b-43af-92c8-51f89b566236 None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:17.289055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925a080-6a7b-43af-92c8-51f89b566236 None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:17.289055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925a080-6a7b-43af-92c8-51f89b566236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:17.322865 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-c925a080-6a7b-43af-92c8-51f89b566236 None None] MFA Rules not processed for user `a2fbe4aab80c483b9fbfc86ff81daa38`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:47:17.324389 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-c925a080-6a7b-43af-92c8-51f89b566236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:17.413793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1473/2919] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:47:17.422538 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:17.425462 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:17.530315 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c536c41-902a-473f-a4e7-18fe88161d7b admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:17.536081 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:17.537270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:17.537733 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:17.538190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:17.553326 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:17.606701 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:17.608432 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:17.636423 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1447/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:47:27.402504 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=50 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:27.625660 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=22 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:37.602533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:37.603096 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:37.603240 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:37.623926 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] MFA Rules not processed for user `807807d370a94aa6a312e51c7a8a2db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:47:37.625780 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:47:37.635879 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1474/2921] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:47:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:47:47.618830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:47:56.518968 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:47:56.573039 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a39272-563b-4ed4-a8bb-e8c63537e245 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:47:56.576844 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:47:56.577492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:47:56.577771 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:47:56.578046 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:47:56.588000 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:47:56.589740 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:47:56.599872 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1448/2922] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:47:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:48:06.594920 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=26 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:48:44.314695 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:44.382364 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d5d20-f18c-4110-91ed-72cbcd0b0bce None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:44.387603 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:44.388244 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:44.388574 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:44.388949 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:44.390986 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:48:44.403697 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:44.405345 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:44.415440 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1475/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:48:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:48:46.096751 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:46.106034 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713e34c-aa66-4fac-b517-a9aaa82a4343 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:46.109822 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:46.110409 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:46.110717 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:46.110989 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:46.112680 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:48:46.176349 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:46.177794 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:46.187598 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1449/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:48:49.412964 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:49.423898 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f630850-5d1d-49a0-92ad-a027466c20b1 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:49.429171 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:49.430415 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:49.430691 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:49.430977 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:49.432688 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:48:49.444374 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:49.446320 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:49.456483 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1476/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:48:53.258689 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:53.260267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:48:53.272842 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99bff-a949-4da9-853b-eada75367567 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:53.277915 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:53.278817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:53.278817 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:53.279059 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] PATH_INFO: `/v3/users/18d0202c683747c299c3ea857f72c3f5` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:53.280203 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=18d0202c683747c299c3ea857f72c3f5)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:53.282151 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:53.457886 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d0202c683747c299c3ea857f72c3f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:53.458535 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d0202c683747c299c3ea857f72c3f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:53.465917 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d0202c683747c299c3ea857f72c3f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:53.466545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d0202c683747c299c3ea857f72c3f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:53.549375 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d0202c683747c299c3ea857f72c3f5'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:53.568087 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1450/2926] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:48:53 2022] DELETE /identity/v3/users/18d0202c683747c299c3ea857f72c3f5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:48:53.869713 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:53.876946 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfec634-c28d-400d-8cdc-747b80d6ff5d service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:53.939398 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:53.940055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:53.940339 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:53.940615 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] PATH_INFO: `/v3/projects/49ae1d33c9874be780d3d55c0d2fc784` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:53.941813 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ae1d33c9874be780d3d55c0d2fc784)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:53.943859 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:54.079529 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ae1d33c9874be780d3d55c0d2fc784', 'request_id': 'req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.083820 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ae1d33c9874be780d3d55c0d2fc784', 'request_id': 'req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.177540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ae1d33c9874be780d3d55c0d2fc784', 'request_id': 'req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.189285 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] The token cache is being invalidate because project 49ae1d33c9874be780d3d55c0d2fc784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:48:54.189572 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.190346 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.191776 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1477/2927] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:48:53 2022] DELETE /identity/v3/projects/49ae1d33c9874be780d3d55c0d2fc784 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:48:54.196826 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:54.198270 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:48:54.212166 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c150a-ffad-4dd4-a41b-757cce7bc9cd admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:54.267841 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:54.268513 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:54.268753 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:54.269067 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] PATH_INFO: `/v3/users/4c81771b1c9c463e9403c90d64e9ea35` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:54.270197 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c81771b1c9c463e9403c90d64e9ea35)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:54.272125 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:54.278545 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:48:54.485144 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c81771b1c9c463e9403c90d64e9ea35'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.487853 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c81771b1c9c463e9403c90d64e9ea35'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.493499 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c81771b1c9c463e9403c90d64e9ea35'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.494189 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c81771b1c9c463e9403c90d64e9ea35'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.563707 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-f9502b44-6f5a-4c24-b036-05729ac0d9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c81771b1c9c463e9403c90d64e9ea35'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:54.582267 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1451/2928] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:48:54 2022] DELETE /identity/v3/users/4c81771b1c9c463e9403c90d64e9ea35 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:48:54.841396 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:48:54.843249 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:48:54.857768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb20cb-e0e4-4b42-ad45-1753e44ba8a8 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:48:54.861611 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:48:54.915503 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:48:54.916274 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:48:54.916393 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:48:54.916618 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] PATH_INFO: `/v3/projects/0f87a3ef42314ffd897888f28aba24be` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:48:54.917793 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f87a3ef42314ffd897888f28aba24be)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:48:54.919689 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:48:55.021390 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f87a3ef42314ffd897888f28aba24be', 'request_id': 'req-df190eb1-2f07-49ca-8b4e-b99a20a182c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:55.028638 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f87a3ef42314ffd897888f28aba24be', 'request_id': 'req-df190eb1-2f07-49ca-8b4e-b99a20a182c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:55.112765 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f87a3ef42314ffd897888f28aba24be', 'request_id': 'req-df190eb1-2f07-49ca-8b4e-b99a20a182c1'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:55.124037 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] The token cache is being invalidate because project 0f87a3ef42314ffd897888f28aba24be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:48:55.124329 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:55.124997 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:48:55.126492 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1478/2929] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:48:54 2022] DELETE /identity/v3/projects/0f87a3ef42314ffd897888f28aba24be => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:49:04.578452 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:49:05.120894 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:49:57.711479 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] REQUEST_METHOD: `POST` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:49:57.712921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:49:57.712921 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:49:57.731656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.auth.core [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] MFA Rules not processed for user `807807d370a94aa6a312e51c7a8a2db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81561) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:49:57.732870 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:49:57.788671 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1452/2930] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:49:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:49:58.103734 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:49:58.105322 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:49:58.166489 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190eb1-2f07-49ca-8b4e-b99a20a182c1 admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:49:58.170003 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:49:58.170610 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:49:58.170887 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:49:58.171161 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:49:58.173537 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:49:58.182891 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:49:58.184351 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:49:58.193166 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1479/2931] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:49:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:50:04.672132 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:50:04.673579 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:50:04.686190 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784e9d1-0f5d-4a98-b92e-0e333a48673a None None] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:50:04.690495 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:50:04.691095 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:50:04.691377 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:50:04.691652 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:50:04.693405 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:50:04.765114 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:50:04.766556 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:50:04.778051 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1453/2932] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:50:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:50:05.961326 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:50:05.970342 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091e010-451b-4c15-bd78-348aafa1e3b6 service neutron] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:50:05.974196 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:50:05.974781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:50:05.975055 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:50:05.975324 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:50:05.977136 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:50:05.990367 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:50:05.991771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03da073-bc16-4176-b83e-23996e1810e4 service neutron] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:50:06.003105 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1480/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:50:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:50:14.772838 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=59 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:50:15.997895 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=32 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:51:27.098860 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:51:27.108001 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c3d5e-c82a-4b54-b0d8-625e4fa07d02 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:51:27.111667 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:51:27.112289 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:51:27.112566 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:51:27.112836 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:51:27.114626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:51:27.125979 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:51:27.127411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:51:27.136076 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1454/2934] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:51:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:51:37.131641 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:51:52.239771 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] REQUEST_METHOD: `POST` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:51:52.240309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:51:52.240309 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:51:52.259438 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.auth.core [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] MFA Rules not processed for user `807807d370a94aa6a312e51c7a8a2db2`. Rule list: `[]` (Enabled: `True`). {{(pid=81560) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:51:52.260612 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:51:52.267826 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1481/2935] 10.222.0.42 () {58 vars in 958 bytes} [Tue Oct 4 10:51:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:51:57.079464 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:51:57.081678 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:51:57.097931 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011d923-a952-4569-960e-151844da39e1 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:51:57.105419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:51:57.105419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:51:57.105419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:51:57.105419 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:51:57.106679 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:51:57.120348 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:51:57.122154 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:51:57.134120 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1455/2936] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:51:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:52:02.254243 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=3 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:52:07.129306 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=6 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:52:10.961340 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:52:10.964500 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:52:10.980540 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c3acc-29c5-44ab-9957-ced0398ef34e None None] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:52:10.987380 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'adc1b4c4a2de4d97bb8019c9efc05570', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:52:10.988257 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:52:10.988545 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:52:10.988830 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:52:10.992781 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:52:11.009745 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:52:11.012621 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:52:11.026699 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1482/2937] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:52:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:52:21.021982 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=6 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:53:42.300052 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:53:42.319976 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b17e8d2-49a9-4416-acc5-b5cc35296fb0 service nova] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:53:42.323626 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:53:42.324216 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:53:42.324492 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:53:42.324763 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:53:42.326505 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:53:42.361374 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:53:42.363408 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:53:42.372224 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1456/2938] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:53:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:53:46.721090 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:53:46.723108 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:53:46.866645 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea819c7c-0028-413c-ae70-46547512362f service nova] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:53:46.871938 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:53:46.872543 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:53:46.872832 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:53:46.873151 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:53:46.874903 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:53:46.886671 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:53:46.888280 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:53:46.899208 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1483/2939] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:53:52.368185 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=21 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:53:56.893992 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=63 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:53:57.665787 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:53:57.684714 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aec54-3a99-47fd-89b3-2605154fcad6 service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:53:57.688471 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '204544f0b58042c7957186df4bdd3949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:53:57.689119 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] REQUEST_METHOD: `GET` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:53:57.689399 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:53:57.689674 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:53:57.691829 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:53:57.777138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:53:57.778574 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:53:57.797457 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1457/2940] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:54:01.247230 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:54:01.248720 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:54:01.262375 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33891e-a888-46c1-9b97-e3e2ddf5ab7e service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:54:01.267510 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a172b6b42d6443029ce492937796e2fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7383c7c14750435da1af9814d3b13f73', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:54:01.268111 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] REQUEST_METHOD: `GET` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:54:01.268392 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:54:01.268684 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:54:01.270464 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:54:01.282078 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:54:01.283711 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:54:01.294754 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1484/2941] 10.222.0.42 () {60 vars in 1365 bytes} [Tue Oct 4 10:54:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:54:04.985260 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:54:04.986923 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:54:04.999904 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffdb6f-192a-4b3e-8563-97cdf8f773ed service neutron] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:54:05.004778 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:54:05.005434 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:54:05.005710 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:54:05.005978 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] PATH_INFO: `/v3/users/807807d370a94aa6a312e51c7a8a2db2` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:54:05.007091 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=807807d370a94aa6a312e51c7a8a2db2)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:54:05.009068 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:54:05.206593 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807807d370a94aa6a312e51c7a8a2db2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.207167 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807807d370a94aa6a312e51c7a8a2db2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.214311 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807807d370a94aa6a312e51c7a8a2db2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.214971 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807807d370a94aa6a312e51c7a8a2db2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.290028 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807807d370a94aa6a312e51c7a8a2db2'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.308341 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1458/2942] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:54:04 2022] DELETE /identity/v3/users/807807d370a94aa6a312e51c7a8a2db2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:54:05.542844 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:54:05.554333 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b8ddd-675a-4e88-a483-b139d1057607 service placement] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:54:05.614798 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:54:05.615445 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:54:05.616198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:54:05.616198 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] PATH_INFO: `/v3/projects/90a1d65daca849b88e1054b8ea48e0e0` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:54:05.617195 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a1d65daca849b88e1054b8ea48e0e0)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:54:05.619122 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:54:05.753837 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a1d65daca849b88e1054b8ea48e0e0', 'request_id': 'req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.760291 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a1d65daca849b88e1054b8ea48e0e0', 'request_id': 'req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.836736 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a1d65daca849b88e1054b8ea48e0e0', 'request_id': 'req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.848554 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] The token cache is being invalidate because project 90a1d65daca849b88e1054b8ea48e0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:54:05.848843 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.849551 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:05.851028 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1485/2943] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:54:05 2022] DELETE /identity/v3/projects/90a1d65daca849b88e1054b8ea48e0e0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:54:05.855278 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Authenticating user token {{(pid=81561) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:54:05.856719 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.fernet_utils [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81561) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:54:05.869467 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a35cf-2223-4e18-8d2a-93efe64c5671 admin admin] Validating token access rules against request {{(pid=81561) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:54:05.896422 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:54:05.927337 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81561) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:54:05.927959 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:54:05.928236 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] SCRIPT_NAME: `/identity` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:54:05.928506 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] PATH_INFO: `/v3/users/a2fbe4aab80c483b9fbfc86ff81daa38` {{(pid=81561) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:54:05.929656 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2fbe4aab80c483b9fbfc86ff81daa38)` {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:54:05.932900 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] RBAC: Authorization granted {{(pid=81561) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:54:06.092158 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2fbe4aab80c483b9fbfc86ff81daa38'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.092911 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2fbe4aab80c483b9fbfc86ff81daa38'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.100411 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2fbe4aab80c483b9fbfc86ff81daa38'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.101138 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2fbe4aab80c483b9fbfc86ff81daa38'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.170134 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG keystone.notifications [None req-ee61faf2-d17b-4dc5-be3d-1abe2326935e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2fbe4aab80c483b9fbfc86ff81daa38'} {{(pid=81561) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.192325 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: [pid: 81561|app: 0|req: 1459/2944] 10.222.0.42 () {58 vars in 1224 bytes} [Tue Oct 4 10:54:05 2022] DELETE /identity/v3/users/a2fbe4aab80c483b9fbfc86ff81daa38 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:54:06.531533 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Authenticating user token {{(pid=81560) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:54:06.535075 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.fernet_utils [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81560) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:54:06.546722 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ed7f6-bca1-4f60-8ed9-e03f8117417d admin admin] Validating token access rules against request {{(pid=81560) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:54:06.604819 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105ad03a7a5c40c78500f0ee01583fae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '76254fdd7dd646c69803abc10a472bc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81560) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:54:06.605486 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:54:06.605768 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:54:06.606048 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] PATH_INFO: `/v3/projects/8875fee8cc474f3d89ba073ffc19053e` {{(pid=81560) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:54:06.607263 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8875fee8cc474f3d89ba073ffc19053e)` {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:54:06.609289 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] RBAC: Authorization granted {{(pid=81560) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:54:06.630901 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:54:06.729043 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8875fee8cc474f3d89ba073ffc19053e', 'request_id': 'req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.735181 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8875fee8cc474f3d89ba073ffc19053e', 'request_id': 'req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.842827 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8875fee8cc474f3d89ba073ffc19053e', 'request_id': 'req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9'} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.854126 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] The token cache is being invalidate because project 8875fee8cc474f3d89ba073ffc19053e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81560) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:54:06.854427 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.855061 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG keystone.notifications [None req-de1d9a6f-b5d3-48f5-9711-d4384c66c1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81560) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:54:06.856544 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: [pid: 81560|app: 0|req: 1486/2945] 10.222.0.42 () {58 vars in 1233 bytes} [Tue Oct 4 10:54:06 2022] DELETE /identity/v3/projects/8875fee8cc474f3d89ba073ffc19053e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:54:16.188173 dv-cinder-859193-3-iscsi devstack@keystone.service[81561]: DEBUG dbcounter [-] [81561] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81561) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:54:16.922536 dv-cinder-859193-3-iscsi devstack@keystone.service[81560]: DEBUG dbcounter [-] [81560] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81560) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}